If you're in a terminal and want to edit just that command (and Vim is your $EDITOR), you can type `ctrl-x ctrl-e` and it will open your command in a buffer. Once you're satisfied, you can `:wq` and the edited command will be ready to execute in your terminal.
Not exactly your usecase, but a useful one nevertheless.
That looks great! I'm not sure it was available in 2018 when I looked. I tried a few options, but posting to twitter & subbing to the account was the quickest hack that worked reliably.
Absolutely! My original plan had been to scrape their website using Selenium or similar.
I quickly noticed that they had employed lazy loading, which would have made that all but impossible. It took me a good few minutes to realise that if they had lazy loading, there had to be a backend, and I was overjoyed when I found out it was serving JSON.
All in all, it was probably much cheaper for them to have me hitting the API endpoint every minute than scraping the website even once a day
Hi! I answered this in the other post, but the Postgres UPSERT was the key. By using a combination of attributes (It may have just been name & D.O.B), In a single operation I could figure out if the cat has been posted (and update the existing row), or create a new row.
This worked as there were only ever 30-50 cats online at one time. If it was a thousand, I'm not sure what I would have done.
Edit: I realise now this was a rhetorical question. Oops!
The primary reason was to learn Elixir, so this was just a well-timed excuse to explore the language (and Phoenix, the web framework).
The secondary reason was that my wife was the main client, and she doesn't respond well to raw JSON. Each tweet would be just the cat's name, photo, and a link to the website. I also did some filtering as certain cats have safety requirements we couldn't meet, e.g. no neighbouring cats, no children)
One of the main issues I had to figure out early on, was "how do I distinguish which cats are new, compared to the previous response?". This was made harder because I couldn't rely on the ordering; occasionally previously-posted cats would have details updated and they would move position. Postgres UPSERT was new (to me, at least) at the time, and it seemed like a very handy way to offload the responsibility. There were never more than 50 cats listed at any one time, so it was reasonable enough to request all the animals at once, and let the database figure out which cats were new, based on a combination of identifiers that would make them unique. I could also filter the updated records to see _what_ had been updated, e.g. the cat had now been rehomed.
Another thing Elixir did really well was the polling mechanism. It's absolutely trivial to spawn a worker that can repeatedly perform a task and asynchronously hand it off to be processed.
A few years ago, my wife and I decided to adopt a rescue cat from battersea.org.uk. However, it was a frustrating experience as the staff didn't always update the website regularly, and we'd find that any suitable cats would be snapped up before we'd even seen them.
I spotted that the website served its data to the frontend via an unsecured internal JSON API, so I built an Elixir app that would poll the API endpoint and upsert the cat data into the database. Any new records would get posted to a twitter account (a free way to get notifications on my phone).
It worked beautifully, and when a black cat called "Fluff" popped up, we both knew he was the right one, and we were able to phone them and arrange a meeting before anyone else. Fast forward five years, and he's sitting next to me on the sofa right now, purring away.
Not exactly your usecase, but a useful one nevertheless.