I’ve been really impressed with OpenAI embeddings + vector searches for some full document searches compared to something like standard elasticsearch over a large body of text. Something I think I’m personally missing from the ChatGPT/GPT4/LLM conversation with regards to information retrieval is nested/graph hierarchies.
An example from a previous job where we used a hand tooled NLP system was querying for doctors/dentists/optometrists and being able to take something like “dentist near me who is available in the afternoons and speaks spanish”. We would parse this user query into a few different queries that would run against a search cluster and database to return the filtered result set, or the closest output.
What would be the ideal way to prepare or tokenize this data for querying with an LLM? It’s partially text (match dentist, speak Spanish), partially geographic (near me, doing a geo radius of N miles from providers location, and part filtering (who meets all those criteria and has availability in a time frame). Is this a use case for large token sizes to be able to take in all possible providers? Or parsing a query more easily from human language -> SQL/other data store query language? Or perhaps figuring out another way to encode this data?
Two different streams that have built up over time. I spend less than 10 hours combined weekly on them averaged over a month if i had to guess but both were independent full time jobs at one point
1) ($$m total, $$k/mo now) Server hosting! This started off as hosting game servers for friends in early high school. It expanded to friends of friends for awhile before I pivoted more towards crypto mining (yes I know, young kid with dubious ambitions). Lots of my first software experience was here. I wrote some software for switching the processes between mining/game hosting across the different boxes (raw processes on servers, the horror). Nowadays it’s winding down as just game hosting and some scientific computing rental to a few universities/their robotics clubs. It’s still slightly profitable but I have no interest in updating servers (CPUs from 2013-2015, GPUs mostly re-sold except for what a few people requested) and it runs everything via containers now.
2) ($$k/mo) Sports film review. I wrote the first version of this my first year of college which was a way to keep the stats book for basketball and football games and stitch the actions with the video footage. We had customers throughout ~20 states primarily high school but some colleges as well. In fact it still runs at a lot of them, but I’m not really connected with managing of it anymore. My co-founder still runs it and we rotate a few students from our alma-matter in as interns and occasionally juniors on it.
This became the basis for an esports version of the software that I created a few years ago. This time with CV to do all the gathering of stats and allowing for jumping around in videos and analyzing overall stats from the output. This started in Call of Duty for their then new professional league but expanded out to Halo, Rocket League and Valorant since. I still do some occasional retraining of the models but the product itself got acqui-hired by a larger company for which I still “consult”
EDIT: I’ve also had many more that cost me more money than they ever made, but I’m a big proponent of failing fast and iterating
IMO, the release of Warzone 1, Call of Duty’s Battle Royale mode that released a few months after Modern Warfare 2019, has completely changed the priorities and incentives for the company. The goal is to sell in game cosmetics more so than the game itself, as many other games are turning to as well.
The yearly release cycle, which may be ending soon, leads to bugs that re-emerge each year and features that appear and disappear. Sure, the different studios which produce the games need some room to innovate, but the inconsistent base set of features is incredibly frustrating. CoD games are one of the games I play the most, with the other being a game which is the complete opposite, Old School RuneScape, that has been built on for ~20 years.
I play the current game MWII with friends a few hours a day most days of the week. Multiple times per session my game crashes at random, something I can’t remember with any other major games with a top of the line PC. Like many other pieces of software, chasing other revenue sources seems to have made the quality of the product take a nose dive, with consequences yet to be seen. This is disappointing to me as someone who enjoys playing the game with friends, who has competed in open-bracket events at major tournaments over a span of a few years, and worked directly with the professional league and teams (CDL and CWL) for analytics and software.
I would love to see something akin to an M-based Mac as a dev board of sorts, but I’m fairly sure it would never happen. My home server has transitioned over time from various 4u rack mount options, to a 2u and an M1 Mac Mini, to a Mac Studio with a disk enclosure.
I even tried using a Snapdragon phone dev board for awhile for something more than an RPi
I switched jobs recently and became the defacto DevOps person so have been able to deploy mostly how I want. I’ve used kubernetes at multiple jobs, side projects and at home but for a cost and time constrained startup we are leveraging ECS/Lambda/Batch/Cloudfront. B2C application, mostly low traffic with nearly no traffic off hours. Occasionally we’ll get a big rush, 2 to 3 orders of magnitude more traffic than usual, from a marketing push and haven’t ran into any issues yet.
I still run KEDA at home for managing plex, home assistant, some game servers and other of my own projects. But being the only one who is using the cluster is a different use case than getting RBAC, ingress and management set up correctly for a production cluster IMO. I’ve never had the sole responsibility or permission over a cluster before, so it was a daunting step I decided not to take for my own sake
Yes Pokémon Legends: Arceus was a tipping point for me. The game does not look good at all to me, and across 3 different switches I tested, it felt like it was hitching and not loading correctly even with the low detail and resolution in docked mode. I’m handheld it’s a bit better, but that’s not really how I want to play the game.
On the other hand, my most played game is by far Old School RuneScape, which is a restore of a backup of RuneScape from 2007. The game has certainly evolved in style and content since the re-release, and I use a 3rd party client with an “HD” plug-in which adds more render distance, shadows, lighting and anti-aliasing.
Like the others have said, definitely seems to be hit or miss. I inherited an old cluster running on elastic cloud, migrated it to AWS managed and it reduced monthly costs by nearly an order of magnitude on top of going from a 5-15 minute outage per day for master node election to having 3 over 2 years.
We’ve rewritten most of the interactions with the older version of ES into a new service and use a self-managed OpenSearch cluster on Graviton instances and it’s the most stable Elasticsearch/Solr solution I’ve ever interacted with
Even better then I would have thought, thanks! I’ve primarily dealt with Elasticsearch/Solr in my career, so jumping into the way searches work in Postgres with various native support and plugins is, interesting to say the least. Maybe I just need to break my Lucene roots
Poking a bit at the documentation, I see like/ilike but are there plans for text search/trigram capabilities? Recently I’ve been working with lots of different entities that have searchable properties and exploring searching across different elasticsearch indexes to do “JOIN” like operations but have been exploring Postgres (and related) solutions for better “JOIN” support out of the box
An example from a previous job where we used a hand tooled NLP system was querying for doctors/dentists/optometrists and being able to take something like “dentist near me who is available in the afternoons and speaks spanish”. We would parse this user query into a few different queries that would run against a search cluster and database to return the filtered result set, or the closest output.
What would be the ideal way to prepare or tokenize this data for querying with an LLM? It’s partially text (match dentist, speak Spanish), partially geographic (near me, doing a geo radius of N miles from providers location, and part filtering (who meets all those criteria and has availability in a time frame). Is this a use case for large token sizes to be able to take in all possible providers? Or parsing a query more easily from human language -> SQL/other data store query language? Or perhaps figuring out another way to encode this data?