I have a slightly different view of this. Here are the key reason why Indian's are succeeding:
1. Speaking the facts does not exist in India. Everyone sugarcoats. The successful see-through sugar coating. They are able to see through the true intentions of their peers, allowing them to be better leaders.
2. Indian's do Jugaad from birth. It is a need, as many don't even have stable electricity to light up a study lamp. They find creative ways to live their lives. This Jugaad at an early age gives them the "Power of Unlimited Thinking". A critical component in being a visionary leader. Finding solutions where no one thought it was possible.
This is a groundbreaking achievement by humankind. Even greater than sending people to the moon. Imagine, every time you ate pork, someone received a kidney transplant.
I did work on making a database myself, and I must say that querying 100TB fast, let alone storing 100TB of data, is a real problem. Some companies (very few) don't have much choice but to use a DB that works on 100TB. If you do have small data, then you have a lot of options. But if your data is large, then you have very few options. So it is correct to be competing on how fast a DB can query 100TB of data; while at the same time being slow if you have just 10GB of data. Some databases are designed only for large data, and should not be used if your data is small.
So what we understand from our study is the following:
1. You must have an Azure account with an API key for using their AutoML. An AutoML environment won't get created without a valid key. Both local and cloud runs mandate this.
2. Getting an Azure account requires a credit card on file and comes with a limited-time free trial. This is a big NO NO when it comes to software claimed to be for free use.
3. The free for life services do not call out AutoML anywhere. They do not claim the AutoML environment (a required step) to be free in any form. Check this: https://azure.microsoft.com/en-gb/free/
4. When they say "local" what are they referring to? Run locally on an Azure Notebook, or can I run this locally on my laptop? We have tried and failed ever to get this to run locally on a laptop. So it is not clear whether this is even possible, or the term "local" is misleading.
Have you managed to run Azure ML locally on a laptop, without requiring a connected Azure account?
Yes, if we run the AutoML from our laptop, it uses the API key to create a cloud instance. The data gets uploaded to the cloud and runs on the cloud. Results are thrown back to the local code. We would not call this a local run.
The question is, have you managed actually to use your computer's local resources for training? If so, please do share how this was possible. We would like to know how this was achieved.
Thanks Mark. Please do share your experience. Very keen on hearing how you find our project. We are still in beta, so lots of scope for improvement. We are open to any suggestions you might have.
That is not true actually. You must purchase an Azure VM for training a model. Deploying trained models on your own infrastructure is permitted. Reasonably speaking we would have to call this an enterprise solution, as there is no way to get a trained model without paying Azure fees. Most models will require GPU, and it is not like we can run AzureML on the free GPU offered by Google Colab.
Thanks for your question. Yes, we did research the space a lot before making AutoAI. Here is what we found:
PyCaret: Semi-automatic. You do the first run; then you figure the next set of runs. Ensemble models require manual configuration.
Tpot: Does a great job. Generates 4-5 lines of py code too. But does not support Neural Networks / DNN. So works only for problems where GOFAI works.
H2O.ai: They have an open-source flavor, but the best way to use it is the enterprise version on the H2O cloud. The interface is confusing, and the final output is black-box.
Now there are many in the enterprise category, such as DataRobot, AWS SageMaker, Azure etc. Most are unaffordable to Data Scientists unless your employer is sponsoring the platform.
AutoAI: This is 100% automated. Uses GOFAI, Neural Networks and DNN, all in one box. It is 100% White-box. It is the only AutoML framework that generates high-quality (1000s of lines) of Jupyter Notebook code. You can check some example codes here: https://cloud.blobcity.com
Hi HN, we have seen a lot of AutoML frameworks out there. As a Data Scientist myself, I have refrained from using these because at the end of the day, you have to submit complete source code to your clients, not just a functioning model. That is why we created AutoAI. Given data and target (value to predict), it can automatically discover and fully train the best performing AI solution. Still, most importantly, it also goes on to produce high-quality Jupyter Notebook code. AutoAI does Whitebox AutoML. A much-needed feature for Data Scientists. Do give it a try, and let me know what you think.
The first thing I would do is check if your Github repository is showing the traffic source. If your primary documentation is on Github, then you would expect most people to visit it in order to figure out how to use your software.
Yes absolutely. Jupyter Notebooks are a classic example of success in this space. Many Data Scientists use Jupyter over the browser for day to day work. Including me.
Advantages: Allows you to use server resources than be limited by your laptop compute capabilities. Log in anywhere, means I don't have to carry my laptop around. As long as I have a browser, I get the same environment to work. Mental peace that my work and data are always safe. I can drop my laptop in a swimming pool and yet I will not even lose the last character I typed on that presentation I was making.
Disadvantages: Yeah, I cannot access my system while on an aeroplane. Can't think of anything else honestly, unless you live in a zone where the internet is as bad as on aeroplanes.
All in all, prefer the browser-based virtual machine any day. The advantages outweigh the single disadvantage of the need for a stable internet connection.
Very interesting. Have you considered the possibility of monetising encrypted backups? Allow people to backup their node on a cloud service you provide. Keep the data encrypted in a manner that only the owner can decrypt. This would give users mental peace, and could be a monetisation source for you.
Got it. I see a classic launch problem coming your way. How do you attract the first set of content creators, when there aren't any content consumers? Have you given this a thought?
We actually did work on this a few years ago but did not get enough takers for it. We created a one size fits all database, that leverages the full capability of the file system.
PS: I am the chief architect of the DB, and the project is no longer being actively maintained by us. But if you make a contribution, we will oblige to review and merge a PR.
Bottom line, nothing you do can make your database faster than the filesystem. So why not make a database that just uses the filesystem to the fullest, than creating a filesystem on top of a filesystem. BlobCity DB does not create a secondary filesystem. It dumps all data directly to the filesystem, thereby giving peak filesystem performance. This is scientifically really the best it gets from a performance standpoint. Not necessarily the most efficient in data storage / data-compression standpoint.
This means, we gain speed, while compromising on data-compression. We produce a larger storage footprint, but are insanely fast. Storage is cheap, compute isn't. So that should be okay I suppose.
You make a good point. But there is one more reason for advertising than just conversions. If Coca Cola were to stop advertising, it would free up substantial ad space allowing for their competition to advertise cheap. These large brands don't just advertise for branding/conversions. They plan their spend budget so as to actually influence the bid rate. This makes it harder for newer brands to compete and get any viable ad exposure at affordable prices.
On the same basis, it would also help if you could provide a comparison between GPUs commonly used for ML. Tesla k80, P100, T4, V100 and A100. How has the architecture evolved to make the A100 significantly faster? Is it just the 80GB RAM, or there is more to it from an architecture standpoint?
1. Speaking the facts does not exist in India. Everyone sugarcoats. The successful see-through sugar coating. They are able to see through the true intentions of their peers, allowing them to be better leaders.
2. Indian's do Jugaad from birth. It is a need, as many don't even have stable electricity to light up a study lamp. They find creative ways to live their lives. This Jugaad at an early age gives them the "Power of Unlimited Thinking". A critical component in being a visionary leader. Finding solutions where no one thought it was possible.