lil_tee

vor 5 Jahrendiscuss

Thanks. The 13F XML is not too complicated, here is the relevant section of code in case helpful: https://github.com/toddwschneider/sec-13f-filings/blob/main/...

I haven't worked with XBRL though, so am not sure if it's more involved

lil_tee

vor 5 Jahrendiscuss

GitHub repo available here: https://github.com/toddwschneider/sec-13f-filings

lil_tee

vor 7 Jahrendiscuss

nytimes.com is #3 domain of late, really #2 behind only github.com if you exclude ycombinator.com (all of the Shows, Asks, etc.)

Top 10 domains by # of items on front page since 1/1/18:

   rank |     domain      | count 
  ------+-----------------+-------
      1 | github.com      |  2041
      2 | ycombinator.com |  1911
      3 | nytimes.com     |  1818
      4 | bloomberg.com   |  1028
      5 | medium.com      |   826
      6 | techcrunch.com  |   735
      7 | theguardian.com |   666
      8 | github.io       |   615
      9 | bbc.com         |   558
     10 | arstechnica.com |   493

lil_tee

vor 7 Jahrendiscuss

FWIW, nytimes.com accounts for ~4% of all posts that appear on the HN homepage: https://toddwschneider.com/dashboards/hacker-news-trends/?q=...

lil_tee

vor 8 Jahrendiscuss

Thanks, agreed this is a good idea and I added query param support here: https://github.com/toddwschneider/stocks/pull/4

lil_tee

vor 10 Jahrendiscuss

TLC just released 6 additional months of trip-level data (Jul-Dec 2015): http://www.nyc.gov/html/tlc/html/about/trip_record_data.shtm...

GitHub repo updated to process additional data: https://github.com/toddwschneider/nyc-taxi-data

lil_tee

vor 11 Jahrendiscuss

Number of bikes in active use does not seem to change with the seasons: https://github.com/toddwschneider/nyc-citibike-data/blob/mas...

The summer 2015 increase corresponds to the permanent expansion in August 2015; I'm not sure why there was a dip in Q1 2015

lil_tee

vor 11 Jahrendiscuss

Yes, the database is all on the machine's local 512 GB hard drive. I did store the downloaded flat text files to an external drive and loaded them into the db from there.

lil_tee

vor 11 Jahrendiscuss

Sure, there are undoubtedly lots of examples of businesses that opened in desolate areas and created new taxi activity where there had been none previously, I just happened to focus on Williamsburg for the post.

Another idea that I didn't get around to doing was to look at concert venues and measure taxi traffic around particular concerts to see if it would correlate to bands' overall popularity

lil_tee

vor 11 Jahrendiscuss

Yep, all on a 2012 MacBook Air. Data size was over 400GB with indexes

Simple queries on indexed columns of the trips table take a minute or two, more complicated queries that require a full sequence scan can take up to a few hours

lil_tee

vor 11 Jahrendiscuss

My dataset does not include livery cabs

FiveThirtyEight has some additional for-hire vehicle data in their GitHub which they obtained via FOIL request: https://github.com/fivethirtyeight/uber-tlc-foil-response/tr...

Intuitively, I would think livery cabs have lost significant market share to Uber, but I don't actually know

Submissions

Show HN: Collect and View Data from SEC 13F Filings

Show HN: SEC 13F Filings – Investment Manager Quarterly Holdings Data

Show HN: Resources to create detailed election results map of NYC

Reverse Engineering Ride-Hail Surge Pricing Trends in Chicago

Reverse Engineering Ride-Hail Surge Pricing Trends in Chicago

Reverse Engineering Ride-Hail Surge Pricing Trends in Chicago

Taxi and Ridehailing Usage in Chicago

Show HN: Hacker News Front Page Trends

Show HN: Hacker News Front Page Trends Search

Taxi and Ridehailing Usage in New York City

NYC Taxi and Ridehailing Usage Dashboard

The Latest Taxi, Uber, and Lyft Usage Stats in New York City

Taxi and Ridehailing Usage in New York City

Mapping Motor Vehicle Collisions in New York City

Using Countdown Clock Data to Understand the New York City Subway

Show HN: A free, lightweight static page to get stock quotes using the IEX API

Assessing Shooting Performance in NBA and NCAA Basketball

Show HN: Interactive NCAA men's basketball shot charts with R and Shiny

Ride-hailing’s NYC dominance, and the impact of DeleteUber

When Citi Bikes Are Faster Than Taxis in New York City

comments