That's super cool. I launched a book recommendations feature this year, which works vastly different. I ask users what their favorite books are(which you can rank), and then allow them to import their goodreads data which includes star reviews, and books they have read, then I determine your favorite style of books based on genres and subjects, then use opensearch to find similar books. It's a lot more complicated than that, but seems to work well. I'm always looking for ideas on how to improve this feature. Interested in what you are actually doing on the backend on the how-it-works page. thanks!
It's much more powerful if you're a member. you can restrict the results to certain genres or book lengths, as we as published date ranges, etc. If someone wants to try out the more powerful feature, DM me and i'll mark your account as a member.
I failed out of college my senior year because I discovered EQ. So many fond memories. I created a guild on a new server and it ended up one of the most famous and best guilds on any of the servers. The amount of planning and management it took to lead a large guild was just ridiculous. It was a full time job. I even created one of the first "loot" web apps in php 3 and mysql just to keep track of player participation and make loot more fair.
Most of the team who created World of Warcraft were members in the guild.
some of my fondest memories:
- getting pretty far in the Plane of Air, which was an incomplete end game zone with almost impossible to beat bosses.
- defeating the Avatar of War, which was not supposed to be killable. We figured out we could charm his guards by having a huge number of enchanters and use the guards to tank him. We managed to beat him and they patched/fixed the guards and made them uncharmable shortly afterwards
The death penalty in the end game zones was originally very tough to work around. You needed a key to reach the zone, but if you died the key required to get into the zone was on your corpse inside the zone. So if everyone wiped/died getting everyone's corpse back was a multi hour event.
I think it's most useful for discovering new reads, especially with the advanced search and recommendations functionality. I do agree i could do a better job of non spoiler summaries. good idea
- Series have always been a problem. Some book lists will include the entire series, and then some will have individual books. If the series is sold as a single book I'll often just include that. Like Lord of the Rings. Sometimes I will include only the first book in the series on a list, to prevent always adding every single book in a series when a list mentions "harry potter series".
basically I don't have a perfect way of handling series'
for the last point, kind of. If you add a book to the default "My Favorite Books" user list, it gets aggregated and used for this book list which is included in the rankings. https://thegreatestbooks.org/lists/463
I do plan on open sourcing more of the code over time. I also have started working on other sites using the same algorithm implementation (music, movies, video games)
This has just been a side project over the year generating passive income. I get around 250,000 page views a day, and with ads, memberships, and affiliate links I make around $2,500~ a month.
Tech stack is ruby on rails 8, postgresql 17, opensearch, redis, bootstrap 5.3 hosting on 3 servers on linode.
recently launched book recommendations feature for my books side project that I put a LOT of work into. I might be biased but I think it works well as long as you give it your favorite books.
warning: account required, and the full featured version where you can specify book length, include/exclude genres/subjects, etc requires a membership. if you would like to test it though just e-mail me at [email protected] and I'll mark your account as paid.
that's awesome! I've had many many side projects launched in the past 2 decades, but the only one still going is my books site https://thegreatestbooks.org
I created it 17~ years ago mostly as just a tool for myself and now it gets roughly 8 million views a month.
The hardest part of any side project is actually launching it and making it somewhat production ready. I always spend the vast majority of my time dealing with devops/deployment issues/tasks
yeah the amazon product API has some severe limitations. I have 25,000~ books on my site, and I just don't have enough API calls in a day to keep the prices 100% updated. It's on my todo list to revisit this, but it's low on my priority list. I don't make that much from amazon refs(couple hundred a month)
I will say that I don't think they really defend their TOS too much from my experience. I used to have a cookbooks site for years that used the same affiliate tag i use for my greatest books site, and never had any issues.
I built it in 2008, and have rewritten it twice now since them. I have spent quite a bit of time adding new lists. It's definitely a labor of love and I do spend quite a bit of time on it.
I do use Gen AI now to generate genres, descriptions, and to grab other data. Previously years ago i would just scrape it or manually set it.
Amazon has a nice product API with up to date prices.
I am working on bookshop.org integration. I used to also do barnes & noble. The problem is neither of them have APIs to programmatically search for books, so i have to do complicated scraping. example: https://github.com/ssherman/bookshop-search
I created it in 2008 and have maintained and improved it over the years. I am trying to figure out how to monetize it more. I currently make around $2k a month. I just use adsense and have a paid membership feature through buymeacoffee. I get massive traffic and I'm pretty much the #1 result for anything related to best/greatest books.
It's built with Rails and Postgresql and hosted on 3 linode servers. I get around 250k page visits a day.
I had the same issue with TikTok/ByteDance. They were using almost 100gb of my traffic per month.
I now block all ai crawlers at the cloudflare WAF level. On Monday I noticed a HUGE spike in traffic and my site was not handling it well. After a lot of troubleshooting and log parsing, I was getting millions of requests from China that were getting past cloudflare's bot protection.
I ended up having to force a CF managed challenge for the entire country of China to get my site back in a normal working state.
In the past 24 hours CF has blocked 1.66M bot requests. Good luck running a site without using CloudFlare or something similar.
been my main side project since like 2008. working on goodreads import right now. Always working on improving the algorithm. would love to collab with a data scientist on ways to improve my algorithm. https://github.com/ssherman/weighted_list_rank
This problem is only going to get worse. for my thegreatestbooks.org site i used to just get indexed/scraped by google and bing. now it's like 50+ AI bots scraping my entire site just so they can train a LLM to answer questions my site answers without having a user ever visit my site. I just checked cloudflare and in the past 24 hours I've had 1.2 million bot/automated requests
The amount of AI bots scraping/indexing content is just mind boggling. for my books site https://thegreatestbooks.org, without blocking any bots, I was probably getting 500,000~ requests a day from ONLY ai bots. Claudebot, amazon ai bot, bing ai bot, bytespider, openai. Endless ai bots just non-stop indexing/scraping my data.
Before i moved my dns to cloudflare and got on their pro plan, which offers robust bot blocking, they were severely hurting my performance to the point that I bought a new server to offload the traffic.
It's insane how toxic the js environment is. it seems like if a project is over 6 months old, nothing will work. When I yarn install on an old project, i'm rolling the dice. I had a 2 year old next.js side project i was working on and the amount of work to make it work the latest version with just updating the dependencies and reading the upgrade docs were infinitely more complex than just starting over from scratch.
i definitely see a lot of indexing requests from openai in the logs of my books site. I figured it was just the bing integration. (thegreatestbooks.org)
here's my recommendations feature: https://thegreatestbooks.org/recommendations
It's much more powerful if you're a member. you can restrict the results to certain genres or book lengths, as we as published date ranges, etc. If someone wants to try out the more powerful feature, DM me and i'll mark your account as a member.