GSoC topic: Traceroute visualization

[romain-fontugne]

Operators would like to have a better way to visualize latency increases in upstream networks. We have a lot of traceroute data from RIPE Atlas but we miss a good visualization to show traceroute details.

There is some good tools to get inspired from (tracemon and thousand eyes path viz), having our own would be great so we can integrate it with other tools (e.g. IYP, AS Hegemony, etc..)

Edit: updated the thousand eyes link

[pankaj87072]

hey @romain-fontugne I think this is best suited for me because I have experience with frontend development and data visualization techniques.
It is less complex and straightforward than the ' Selecting RIS/Routeviews Collectors'

[pankaj87072]

so can I go ahead with this if someone else is not working on it??

[romain-fontugne]

There is hundreds different ways to implement this. This is one of the reason why this is a GSoC project and not a simple issue. For GSoC projects you should prepare a proposal explaining how you would design and implement that tool. The actual coding is done during the summer. Please see contributors instructions for more details:
https://github.com/InternetHealthReport/gsoc/

[MAVRICK-1]

@pankaj87072 You can directly raise your issue on IHR WEBSITE

[pankaj87072]

But we should discuss the issue first right if it is valid to give time or not before raising shouldn't we

[pankaj87072]

As i have seen the issue section many issue is about responsiveness of website but i think it is not the priority as replied by @romain-fontugne in many issues

[MAVRICK-1]

If your issue pertains to the website, we can raise it there, where maintainers will review it. You can learn about contributing by following this CONTRIBUTING.md

[pankaj87072]

Ohk thanks for the help

[MAVRICK-1]

@romain-fontugne you want something like this right?

Recording.2024-02-17.205538.mp4

[AdamLucife]

@romain-fontugne i got 3 questions.

1. Is there any API for traceroute data ?
2. you said that to integrate with IYP data .Is there any API for IYP, or we have to query from Neo4j.?
3. And basically what kind of graph you want?

[0xAnon0602]

1. You might look into RIPE Atlas API documentation for it.
2. You can query IYP Neo4j using the Neo4j HTTP API

[AdamLucife]

@0xAnon0602 The API is showing 405 error

[0xAnon0602]

That was just the API endpoint for IYP Neo4j
You might look into this if you want to learn how to query it.

[AdamLucife]

@0xAnon0602 thankyou for your help , but that's a POST request, I guess we need get Request to retrieve data?

[romain-fontugne]

there is two ways to query Neo4j, using the HTTP API mentioned above or using the bolt protocol (there is a js driver for that). Please see the neo4j documentation for that

[MAVRICK-1]

@romain-fontugne if I am not wrong the main aim is to integrate datas of IYP (Internet Yellow Pages) AS (Autonomous System) Hegemony and other datas with a network visualization tool such as TraceMON enables operators to gain deeper insights into network performance and characteristics?

[romain-fontugne]

We want to implement our own traceroute visualization tool. TraceMON is a good example to get inspired from but we want our own implementation for at least two reasons:

1. we may adopt a different design that could be more suitable for the data we have
2. having our own implementation give us more flexibility to extend/modify the code later

[pankaj87072]

@MAVRICK-1 you are right and the main focus is latency visuals .you can get reference from my prototype of it and if you any suggestion I will be glad to hear it.

Screencast.from.24-02-24.11-00-55.AM.IST.mp4

[MAVRICK-1]

You used D3 right? But My question was that we have to integrate with TraceMON or show the route paths using plotyjs.

[pankaj87072]

Yes actually this is a project of my friend and i have made the frontend for them so i used D3.js but yes as i understand the idea we have to get reference from tracemon not to integrate with it and yes we have to build the page using ploty.js

[romain-fontugne]

yes, we don't have to integrate with TraceMON. Plotly is not mandatory if you can do better with D3 then we can use D3 instead of plotly

[romain-fontugne]

On thing I want to point out, is that traceroutes have a source and destination. So for a visualization it would be more intuitive to have source/destination on the left/right part of the graph (like thousand eyes viz) or top/bottom (like traceMON).

[pankaj87072]

@romain-fontugne what i thought is to give a better look(ui) and add onclick box for details like latency,dependency and other thing.what is your views on this.

[romain-fontugne]

Please take a look at tracemon and the thousand eyes documentation to get some more ideas.

[pankaj87072]

@romain-fontugne i got it

[pankaj87072]

Thanks for the help

[m-appel]

Let me add my take on this as well: The goal of this topic is to design and implement a visualization for traceroutes that also leverages some of the other data we have available. We explicitly do not just want to integrate one of the existing tools.

When making a proposal for this topic, think about what information is available within a single traceroute, how this could be enhanced with data from IYP etc., and also the implications of multiple traceroutes.

For example, a tracerout hop contains (most of the times) the following things:

1. A TTL
2. The IP(s) of the responding hosts
3. An RTT

The simplest case is to map an IP to an AS number, but there might be other interesting data points associated with it. The evolution of the RTT over different hops is also interesting.

Then you can think about a more abstract view:

1. Traceroutes between the same source and destination over time (what does change?)
2. Traceroutes from different sources to the same destination
3. And more?

Finally, how can all this data be visualized in a non-confusing way? This is the direction this topic is aiming at.

These are just some examples and should not be seen as a comprehensive list of things we want or need. We do not know what is the best way to visualize this, and I do not know if it is even possible to integrate all this in a reasonable timeframe. The point is: Be creative!

[m-appel]

As Romain wrote, the traceroute data comes from RIPE Atlas, you can find the format in their documentation.

It's not yet clear how/when we fetch this data, currently we are periodically downloading a number of measurements and storing them in Kafka. But the exact source of the data should not be too important for this, only the format.

Data from IYP would be queried with neo4j, like done here.

Data from IHR can be retrieved via the API.

Note that we do not necessarily need data from all these sources.

[SAHELISAHAA]

@m-appel thankyou sir , I guess querying form Neo4js would be better, I am facing one issue in which is how will retrieve the data of the routers from source to destination this main problem is what query should I write , I have basic understanding of Neo4j . Only facing this problem, rest is full cleared. We also have to show number of hops between source and destination?

[m-appel]

[...] how will retrieve the data of the routers from source to destination [...]

Sorry, I don't quite understand your question. The IPs of the routers are in the traceroute. If you mean what data to get from IYP related to those IPs, I suggest taking a look at the public instance and explore what is available that relates to IP nodes.

We also have to show number of hops between source and destination?

Yes, this is one of the integral parts of a traceroute result.

[SAHELISAHAA]

@m-appel I mean from the source data packets are transferred from routers to routers eventually reaches its destination, So I was asking about the Cypher query which will show us the data of the routers from where the packet passed.

[m-appel]

I can not give you a Cypher query, since it is not yet decided what we will include. A potential proposal also does not necessarily has to go into that much detail, it is fine to just include data points from IYP that we could potentially include.

You can play around with IYP via the graphical interface which does not necessarily require Cypher knowledge. There is also a Getting Started there or you can check out the neo4j Getting Started.

[pankaj87072]

Hey @romain-fontugne @m-appel i have completed the ui design of web page that how to looks overall
Where should i share to get some feedback and make it better.

[romain-fontugne]

you can send it by email or on slack

[pankaj87072]

ohk sir and with the ui i am also sendig my proposal just for intial review pls give suggestion if any

[MAVRICK-1]

I have made a demo

Recording.2024-02-29.125350.mp4

here the Link
There more this add and implement this is just a rough demo.
@romain-fontugne @m-appel your views on this

[romain-fontugne]

what's the input data?
I think you should make it more readable. As mentioned above traceroutes have a source and destination, it would be nice if you can reflect these in your visualization

[pankaj87072]

hey anyone working in this,i have some question regarding data integration from iyp.

[romain-fontugne]

No one is assigned yet, this is a discussion. Feel free to ask questions here