Ways to fetch financial data for algo trading(github.com)
github.com
Ways to fetch financial data for algo trading
https://github.com/rbhatia46/Fetching-Financial-Data
14 comments
This is very cool. I’m sending the link off to an investor friend who is programming-savvy, and to some others who aren’t.
I can deal with the auto-complete. A list of fields would help whet the appetite of my less-programming savvy friends. Having a manual or tutorial would help, too. A good number of potential users are retired and might have more of that learning-style.
It would be useful to be able to get a list of the industries in your db so a user could deal with the inconsistencies of that naming data. For example, EIN, for instance, matches to profile.industry == "UtilitiesRegulated Electric" but the detailed info displays "Utilities Regulated Electric”. Others, like DUK, match to the latter. Both of these are on NYSE.
I was doing some simple queries, but now it’s not returning data. Maybe you're getting too much attention from hacker news.
Any thought or intention of adding historical data? And allowing for backtesting of screens?
I can deal with the auto-complete. A list of fields would help whet the appetite of my less-programming savvy friends. Having a manual or tutorial would help, too. A good number of potential users are retired and might have more of that learning-style.
It would be useful to be able to get a list of the industries in your db so a user could deal with the inconsistencies of that naming data. For example, EIN, for instance, matches to profile.industry == "UtilitiesRegulated Electric" but the detailed info displays "Utilities Regulated Electric”. Others, like DUK, match to the latter. Both of these are on NYSE.
I was doing some simple queries, but now it’s not returning data. Maybe you're getting too much attention from hacker news.
Any thought or intention of adding historical data? And allowing for backtesting of screens?
Thanks or the feedback.
Regarding manual or tutorial, I agree that its difficult to manage the fields. I namespace the fields into profile, income statement, balance sheet and quotes so ideally you only have to search [incomeStatement...] and see some fields. But I'll try to list them all out.
Regarding the enums, I thought about that before and wanted to implement auto-complete enums in the language grammar but so it was bit complicated so I decided to kick the can, but I'll circle back around to it. There are only a few string fields and I put in regex matching so you can just do contains `contains "electric"`
What queries didn't return? It's a static site, it shouldn't be affected by the traffic.
The historical data is tough because reporting periods aren't all the same. I don't have a back tester either. I did have a feature before where you can plot any attribute historically, but I didn't find it that useful.
Regarding manual or tutorial, I agree that its difficult to manage the fields. I namespace the fields into profile, income statement, balance sheet and quotes so ideally you only have to search [incomeStatement...] and see some fields. But I'll try to list them all out.
Regarding the enums, I thought about that before and wanted to implement auto-complete enums in the language grammar but so it was bit complicated so I decided to kick the can, but I'll circle back around to it. There are only a few string fields and I put in regex matching so you can just do contains `contains "electric"`
What queries didn't return? It's a static site, it shouldn't be affected by the traffic.
The historical data is tough because reporting periods aren't all the same. I don't have a back tester either. I did have a feature before where you can plot any attribute historically, but I didn't find it that useful.
I didn't realize it was running locally, so I think I was inadvertently thrashing it: running in Chrome with many, many tabs open; on an old machine that was probably running low on memory; and randomly pressing F5 instead of Cmd-Enter (some part of my brain was thinking of a SQL environment I use).
I added price and industry to the default display, used NYSE and Nasdaq data sets, and queried something like filter profile.lastDiv > 3 and profile.industry startsWith "Utilities" and profile.industry contains "Electric". I probably tried to start my query before the data sets completed loading. I don't know if interrupting the data loading or pressing F5 is more correlated with it acting strangely. (This is on the latest Chrome, under OSX 10.13.6.)
I tried again using Firefox 79.0 and it works fine. F5 reloads the page (and logs me out, ahem). Tried to do a few things while it was loading data and it was more solid.
I'd put saving column layouts (at least the last one used) ahead of enums for field matching. Seems like once you add enums, you'll have to be continually vigilant for data oddities.
I think I'd find it interesting to be able to plot attributes or computed attributes against another. Not sure how other tools get around the reporting period issue.
Interesting tool; thanks for putting it out.
I added price and industry to the default display, used NYSE and Nasdaq data sets, and queried something like filter profile.lastDiv > 3 and profile.industry startsWith "Utilities" and profile.industry contains "Electric". I probably tried to start my query before the data sets completed loading. I don't know if interrupting the data loading or pressing F5 is more correlated with it acting strangely. (This is on the latest Chrome, under OSX 10.13.6.)
I tried again using Firefox 79.0 and it works fine. F5 reloads the page (and logs me out, ahem). Tried to do a few things while it was loading data and it was more solid.
I'd put saving column layouts (at least the last one used) ahead of enums for field matching. Seems like once you add enums, you'll have to be continually vigilant for data oddities.
I think I'd find it interesting to be able to plot attributes or computed attributes against another. Not sure how other tools get around the reporting period issue.
Interesting tool; thanks for putting it out.
Cool site, thank you for sharing. Do you use the same resources for pricing as listed on this post's github? I ask b/c some companies display NULL such as Alcoa (AA) for Price Earnings.
Also, the Balance Sheet, Income and Cashflow Statements all link to the same quarterly report web page. As a shameless plug for myself, I built https://Last10K.com that allow users to access individual financial statements & disclosures buried within lengthy 10-K/Q filings. We have an API too; maybe stox.dev can link to Last10K.com for this info?
Also, the Balance Sheet, Income and Cashflow Statements all link to the same quarterly report web page. As a shameless plug for myself, I built https://Last10K.com that allow users to access individual financial statements & disclosures buried within lengthy 10-K/Q filings. We have an API too; maybe stox.dev can link to Last10K.com for this info?
I use financial modeling prep [0] for both real time quotes and the quarterly statements and Trading View for charts. The nice thing about it is it has everything parsed from the statements such that you can compare apples to apples. No need to hunt around in the documents for net income for instance. It also links back to the original documents as well. They also have a very well documented api.
Only thing I'm not happy about is that when I checked a few times, it took them a few days before they had earnings available. That and their screener is limited (which is essentially what I built).
Yours looks like it has similar features. How do your features compare? I couldn't find the API docs. Does your premium package allow for commercial use?
[0] https://financialmodelingprep.com/developer/docs/
[1] https://www.tradingview.com/
Only thing I'm not happy about is that when I checked a few times, it took them a few days before they had earnings available. That and their screener is limited (which is essentially what I built).
Yours looks like it has similar features. How do your features compare? I couldn't find the API docs. Does your premium package allow for commercial use?
[0] https://financialmodelingprep.com/developer/docs/
[1] https://www.tradingview.com/
Missed opportunity to call it stonks.dev , but looks pretty cool!
This looks exactly what I've been looking for! Thanks
OP you should consider writing something similar for finding and using alternative data, given that you work at Rakuten. That could also help you pivot from sell-side to buy-side if that's your eventual goal.
Personally I wouldn't recommend Yahoo Finance price data for trading strategies, but the 10Q and 10K data is good. There are semi-professional and entry-level professional sources like IQFeed which are higher quality and more flexible. You tend to get what you pay for when it comes to financial data (and alternative data as well).
Personally I wouldn't recommend Yahoo Finance price data for trading strategies, but the 10Q and 10K data is good. There are semi-professional and entry-level professional sources like IQFeed which are higher quality and more flexible. You tend to get what you pay for when it comes to financial data (and alternative data as well).
Related to datasets for clustering pharmaceutical stocks:
Dexamethasone Announcement Could Have Made Hedge Funds A Fortune — Alpha Week
https://medium.com/@492727ZED/dexamethasone-announcement-cou...
https://medium.com/@492727ZED/dexamethasone-announcement-cou...
[deleted]
After doing some scraping, next is time for some action and implementing an investment strategy using Python with the data we scraped, here is an implementation of the amazing magic formula by Joel Greenblatt, which is known to generate amazing returns in the long term, find below the implementation in python - https://github.com/rbhatia46/Greenblatt-Magic-Formula-Value-...
I'm learning Python for exactly this purpose (well, and for manipulating the data). Thanks, you've given me a leg up.
[deleted]
It's essentially a stock screener. It lets you write arbitrary queries on over 300 fields and ratios from the firm's accounting statements. Its the opposite of algorithmic, but helps you narrow down the field of stocks you may find interesting (eg. Low PE, high book value, certain industries, etc)
I would love some feedback