A text about recent progress using local Large Language Model llama3.1:8b to answer questions about a data corpus using Retrieval Augmented Generation and Named Entity Recognition.
One of Convier's superpowers is to convert any data into a graph without needing a graph database or writing any code. In this video, I show this superpower used on the Microsoft Defender API. This API can, among other things, be used to identify and understand business email compromise and fraud.
Hi HN! I’ve made this introduction video to the data platform we’re making at Convier (https://convier.no). The target audiences are potential users, customers, IT-departments, and investors, and I would love feedback about what we can do to improve the presentation. It is meant to be put on our website, as well as posted on other social media.
I first looked at relational timelines some years ago (https://github.com/pcbje/relational-timeline), and I've been wanting to apply them on crypto transactions. The idea is to provide a simple overview of how funds are moving to and from an address. Curious if people see obstacles and/or opportunities with such an approach.
Are you referring to node placement or selection/merging? Node placement is done with D3 force directed layout, so I don't think I'll be able to do much there. And again, animation makes it more soggy.
Yes. Animating node placement has a significant performance penalty, so in larger graphs you would either hide edges or everything during the first couple of seconds.
At some point you run out of pixels on your screen to visualize it all at once. My suggestion in your case is to not use a graph at the beginning, but rather grouping nodes based on metrics. Like degree (group all contacts of a node that only has a single contact), communities, or value (e.g. IP's in a subnet).
It's built on top D3 and has the same limits (5k nodes 20k edges should not be a big problem). The "big" here is the ability to combine nodes, for two reasons:
1 Reduce the number of nodes and edges, thus increasing capacity
2 Combine nodes that should be seen together (e.g. alternative spellings and typos), to better deal with the variation-aspect of big data
I see your point, but the goal here was to illustrate the functionality.
I really think this tool can be useful for analysts and others working in "thin client environments" with limited possibility to install tools, but I need to get the message out there. This post was a first attempt at describing it and check interest.
Edit: Download button added to page. Source comes later.