Hi! I've been working as a data scientist/MLE for the past 5 years. The last year as a freelancer, and before that working on various NLP tasks at a hedge fund. My generalist skillset with a specialisation in NLP makes me a perfect fit for startups working on generative AI products. I work highly autonomously and thrive in uncertainty. I am available for 32 hours per week until May, and full time after that.
Hi! I've been working as a data scientist/MLE for the past 5 years. The last year as a freelancer, and before that working on various NLP tasks at a hedge fund. My generalist skillset with a specialisation in NLP makes me a perfect fit for startups working on generative AI products. I work highly autonomously and thrive in uncertainty. I am available for 32 hours per week until May, and full time after that.
They are probably not silly mistakes. Label encoding can be very useful for tree based models when the categories are ordinal, or when there are a high amount of categories.
AFAIK he never revealed his intents, so he could just be doing it 'for the lulz'... That seems more likely to me considering his 4chan-y attitude on his other channel 'Vagrant Holiday'
I’m Nathan, working on subsets.io. We’re building a search engine for data, where your search with your internal dataset and can add matches with one click.
— Context —
The datasets we work with during our analytics/modelling jobs are often lacking. They were generated by internal systems, but in many cases there are important missing attributes, or the entities in the internal dataset are significantly influenced by external factors that are not captured.
Unfortunately, integrating with external data is a major hassle. External data is scattered across many different places (data dumps, api providers, api marketplaces, open data platforms, the web), in all kinds of formats. Integrating it is very time-consuming and requires significant technical skills, which is usually not an analysts/scientist’s core competency. Also, the fragmentation of external data makes exploration very difficult. Furthermore, it’s often unclear if a given external dataset will add value so it can be hard to justify the integration investment.
We want to make data exploration and integration easier by treating it as a search problem. Because a dataset contains many values for a given column, we can make much stronger assertions about types than api’s that work with single values. We currently check basic types, and use language models to infer context for more complex string types. For example, if you upload a dataset with countries and their respective alcohol consumption for a year, our system will recommend to add smoking rates, tariff rates, and crime rates.
It’s obviously a very complex problem, but I think that if we can communicate clearly in case of uncertainty we can make the process a lot easier.
Would love to hear your thoughts.
PS: Feel free to drop me a message at [email protected] if you have any questions or would like to chat
Unfortunately this data is often inconsistent with the visual representation. For example, webshops often list their product as 'InStock' regardless of the actual stock status. Since products are in stock most of the time, you will not find out about this and thus likely extract wrong data in the future.
This was especially apparent when I tried to get my hands on some weights during the early corona pandemic. All webshops were out of stock, but in about 80% of them the schema markup indicated otherwise
I think the conclusion of a death toll estimation exercise may shift if you adjust for expected remaining lifetime.
In Scotland, the mean COVID death age was higher than the average lifetime expectancy[1], which indicates that on average those people would probably live <10 years had they not contracted the disease. The expected remaining lifetime of the average individual in Scotland (if it remains unchanged) is ~37 years, which would reduce the lifetime adjusted death toll by at least 3.7.
There have been a significant amount of excess deaths that are not attributable to COVID [2]. I suspect that such deaths have a lower average age (e.g. due to suicides), than COVID deaths, further reducing the average life years saved due to the lockdowns.
I'm not proposing that we should optimise for life years saved, but I do think that the death of a 86 year old is less tragic than that of a 38 year old, and that we perhaps shouldn't evaluate lockdown vs no lockdown purely based on the percentage of the population which died.
Technologies:
- Data Science: Pandas, Numpy, sentence-transformers, Matplotlib, Scikit-Learn, LLMs, ElasticSearch, Pinecone, PyTorch, nltk, XGBoost
- Data engineering: Dagster, Airflow, Spark, Kafka, BigQuery, Postgres, ElasticSearch, Great Expectations
- DevOps: Docker, Kubernetes, GCP, AWS
- Frontend: React.js, Next.js, TailwindCSS
- Backend: FastAPI, Express
Resume: https://www.linkedin.com/in/nsv-3b1a54234
Email: [email protected]
Hi! I've been working as a data scientist/MLE for the past 5 years. The last year as a freelancer, and before that working on various NLP tasks at a hedge fund. My generalist skillset with a specialisation in NLP makes me a perfect fit for startups working on generative AI products. I work highly autonomously and thrive in uncertainty. I am available for 32 hours per week until May, and full time after that.