Location: Europe, Role Type: Fractional, Remote / Worldwide
Remote: Yes
Willing to relocate: Yes
Technologies: Kafka, Flink, Spark, Cassandra, Scala, Rust / PyO3 / Tokio, Python, Zookeeper, Databricks, Delta Lake, BigQuery, Redshift, Hive, Kinesis, Airbyte, Airflow, Temporal, DBT, Aerospike, Snowflake, PrestoSQL, Trino, Clickhouse, SQL, Vector DBs, Golang, gRPC/protobuf, Terraform, CUDA
Resume: https://drive.google.com/file/d/1lZuSpneLDVzzYoeaA8or-m5m2dfYAMB4/view
Email: in the profile (please do mention "Via HN" in the subject for your email to land in the right place in my inbox)
[Remote Worldwide, Fractional] I'm pursuing a role in the high-scalability distributed systems space. I'm a well-rounded Scala/Rust/Python dev, well-versed in data engineering, with deep knowledge of the internals of distributed datastores. I have experience with data modeling for high throughput database activity and a strong understanding of which workloads and data access patterns are scalable and what datastore the data should reside in. I have a highly confident ability to lead a data / MLops project from start to finish. Hire me as fractional data engineer.
Core Focus Areas:
● Cassandra (Data Modeling, Troubleshooting Performance And Operational Issues)
At some point, I needed to write a function which, given a collection of product titles, picks one that is neither the longest nor the shortest, it should pick the one which best captures the essence of the product while not being excessively verbose. For example, given the product titles below, it should pick "Portable Two-Way Translator, Handheld".
Based on previous experience with centroid-based algos, the function I wrote does a first pass throwing all words from all product titles into one big bag, then computing a centroid (frequency histogram with low-frequency words removed). The second pass is to compute a cosine similarity score for each product title (its own frequency histogram against the centroid). Whichever product title is the most similar to the centroid wins.
That algo may have existed already in some academic paper somewhere, but I came up with it independently.
SEEKING WORK, Senior Data Engineer, Remote / Worldwide
Well-rounded Scala/Rust/Python dev, well-versed in data engineering, with deep knowledge of the internals of distributed datastores. I have experience with data modeling for high throughput database activity and a strong understanding of which workloads and data access patterns are scalable and what datastore the data should reside in. I have a highly confident ability to lead a data / MLops project from start to finish. Hire me.
Core Skills:
● Cassandra (Data Modeling, Troubleshooting Performance And Operational Issues)
I'm not sure if I should be applying but before I do, I'm wondering what kind of novel or interesting data driven algorithms you refer to here. Are you referring to like adaptive query planners based on runtime statistics? Real-time detection of query latency spikes? Automated creation of secondary indexes based on usage patterns? Statistical smarts to decide what to cache and when? These would be sort of runofthemill for any modern database product, can you clarify if you are looking for any of these?
[Remote Worldwide, Fractional] I'm pursuing a role in the high-scalability distributed systems space. I'm a well-rounded Scala/Rust/Python dev, well-versed in data engineering, with deep knowledge of the internals of distributed datastores. I have experience with data modeling for high throughput database activity and a strong understanding of which workloads and data access patterns are scalable and what datastore the data should reside in. I have a highly confident ability to lead a data / MLops project from start to finish. Hire me as fractional data engineer.
Core Focus Areas:
● Cassandra (Data Modeling, Troubleshooting Performance And Operational Issues)
● Apache Iceberg (Scaling, Tuning, Self-Hosted Setup)
● Stream Processing At Scale: Kafka, Flink, Spark Streaming, Storm
● Custom-Crafted Contextualized Embeddings, Vector-Based Semantic Search, Deep Intent Recognition In Search Engine Queries
● Languages: Scala, Rust, Python, SQL (proficient), Golang (ramping up)
Educational Background: Computer Science.
Solid experience working remotely and working with teams that are distributed geographically. I typically work Pacific Time hours.
Ask: $385K base, pro-rated.