akbar501

vor 2 Jahrendiscuss

Was about to say the exact same thing. Housing constraints are not limited to "near Nvidia HQ". It's the entire Bay Area.

For those not from the area, it's city after city after city in the Bay Area. 60 miles up the peninsula (from San Jose to San Francisco), and 60 miles up the East Bay, the vast majority of which has inventory and affordability challenges.

akbar501

vor 5 Jahrendiscuss

TL;DR Give Blitz a try.

I've been developing my app with Blitz. It has been an extremely productive framework to work with.

The following are a few of the productivity boosts that I like:

1. It includes authentication plus there's a good library for authorization (blitz-guard) so auth is a breeze.

2. The generator is helpful when I started with Blitz, but use it less now.

3. The community is friendly and helpful. ex. Check out this post on skeleton loaders https://andreas.fyi/engineering/nextjs-auth-skeleton-loaders

4. The integration with Prisma is buttery smooth. In your UI code you call a function that has the database access. Blitz converts the function calls into API calls and runs your database code on the server. When Prisma is not enough you can drop to `db.$queryRaw` to write SQL directly.

5. Personally, I dislike fiddling with 800 libraries to get a project up and running. Blitz includes Jest, Prettier, and everything else you need so on so that you're productive from day 1.

akbar501

vor 6 Jahrendiscuss

One request I would have for SUS is to add the ability to add a weekly update in the past.

More than once I was working late Sunday and well into Monday morning. When this happens I can no longer provide an update for the prior week so it looks like I've done nothing for that week.

akbar501

vor 8 Jahrendiscuss

Tenable | SF / Bay Area | ONSITE or REMOTE | Sr. Software Engineer and Principal Engineer positions open

Tenable is a rapidly growing network security company. We’re expanding our engineering team to keep up with our rapid customer and revenue growth.

Tenable.io is the first comprehensive cyber exposure platform. You will be joining a team that is building our data management capabilities, streaming data pipelines using Kafka, ingest APIs, and various other parts of our data infrastructure. This is a great opportunity to have a significant impact.

In terms of skills, you should be able to develop, deploy and maintain a microservice written in Java/Kotlin or another JVM language, experience with databases (PG, C*, Elastic, etc), and expertise managing data at scale. If you already know Kafka that's a plus. AWS, GCP or Azure experience is needed.

I'm the Sr. Director of Engineering, Data Services. You can reach me at aahmed @ tenable.com. Please put "HN" in the subject line.

akbar501

vor 8 Jahrendiscuss

Tenable | SF / Bay Area | ONSITE or REMOTE Software Engineer, Principal Engineer

Tenable is a rapidly growing network security company. We’re expanding our engineering team to keep up with our rapid customer and revenue growth.

Tenable.io is the first comprehensive cyber exposure platform. You will be joining a team that is building a streaming data pipeline using Kafka and Java/Kotlin. We are also building connectors to 3rd party applications using a FaaS (aka serverless). This is a great opportunity to have a significant impact.

In terms of skills, you should be able to develop, deploy and maintain a microservice written in Java/Kotlin or another language that runs on the JVM, that exposes a REST API, calls other REST APIs, parses and produces JSON, reads/writes to/from a data store (PG/Aurora...). It's a plus if you understand how to instrument your code (telemetry, logs, etc.), and ideally understand retries with backoffs, ideally circuit breakers, etc. If you already know Kafka that's a plus. AWS, GCP or Azure experience is needed.

I'm the Sr. Director of Engineering, Data Services (Data Science and Data Engineering). You can reach me at aahmed @ tenable.com. Please put "HN" in the subject line.

akbar501

vor 8 Jahrendiscuss

Tenable | SF / Bay Area | ONSITE or REMOTE Software Engineer, Sr. Software Engineer, Principal Engineer, Engineering Manager

We're hiring at all skill levels.

Tenable is a rapidly growing network security company. We’re expanding our engineering team to keep up with our rapid customer and revenue growth.

Tenable.io is the first comprehensive cyber exposure platform. You will be joining a team that is building a streaming data pipeline using Kafka and Java/Kotlin. This is a great opportunity to have a significant impact.

In terms of skills, you should be able to develop, deploy and maintain a microservice written in Java/Kotlin or another language that runs on the JVM, that exposes a REST API, calls other REST APIs, parses and produces JSON, reads/writes to/from a data store (PG/Aurora, or whatever). If you're applying to one of the more senior roles, then you should also understand how to instrument your code (telemetry, logs, etc.), and ideally understand retries with backoffs, ideally circuit breakers, etc. If you already know Kafka that's a plus. AWS, GCP or Azure experience is needed.

Interview process

We interview quickly. Our goal is 5-10 business days for the entire process.

  - Cultural fit with Director Engineering (i.e. me)
  - Technical interview with 2 peers
  - Coding challenge (practical exercise similar to what you'll actually do)
  - Decision

I'm the Director of Engineering, Ingest and Pipelines. You can reach me at aahmed @ tenable.com. Please put "HN" in the subject line.

akbar501

vor 8 Jahrendiscuss

Tenable | SF / Bay Area | ONSITE or REMOTE Software Engineer, Sr. Software Engineer, Principal Engineer, Engineering Manager

We're hiring at all skill levels.

Tenable is a rapidly growing network security company. We’re expanding our engineering team to keep up with our rapid customer and revenue growth.

Tenable.io is the first comprehensive cyber exposure platform. You will be joining a team that is building a streaming data pipeline using Kafka and Java/Kotlin. This is a great opportunity to have a significant impact.

In terms of skills, you should be able to develop, deploy and maintain a microservice written in Java/Kotlin or another language that runs on the JVM, that exposes a REST API, calls other REST APIs, parses and produces JSON, reads/writes to/from a data store (PG/Aurora, or whatever). If you're applying to one of the more senior roles, then you should also understand how to instrument your code (telemetry, logs, etc.), and ideally understand retries with backoffs, ideally circuit breakers, etc. If you already know Kafka that's a plus. AWS, GCP or Azure experience is needed.

Interview process

We interview quickly. Our goal is 5-10 business days for the entire process. - Cultural fit with Director Engineering (i.e. me) - Technical interview with 2 peers - Coding challenge (practical exercise similar to what you'll actually do) - Decision

I'm the Director of Engineering, Ingest and Pipelines. You can reach me at aahmed @ tenable.com. Please put "HN" in the subject line.

akbar501

vor 9 Jahrendiscuss

Amazon is a huge threat for any infrastructure company using Apache 2.0. If you gain popularity, then Amazon will be a direct competitor once they host your project. Given that Amazon's services benefit from the IE effect, it's not irrational for an open source infrastructure company to eliminate such a threat via licensing.

akbar501

vor 9 Jahrendiscuss

It's comparable to LevelDB and RocksDB. However, Badger is not comparable to Cassandra, ScyllaDB or DynamoDB as they are distributed database management systems.

Cassandra, for example, has it's own storage engine that's responsible for writing bits to disk.

akbar501

vor 9 Jahrendiscuss

Redis is a database management system (DBMS). Badger is a storage engine.

An application developer would not choose Badger, but instead would pick a DBMS such as Redis.

A database engineer would use Badger to develop a DBMS that the application engineer could use. If the database engineer so chooses they could expose a Redis compatible API.

akbar501

vor 9 Jahrendiscuss

The terminology can be a bit confusing. Database Management System is what people often call a database. Unfortunately, the storage engine is also called a database.

Badger, RocksDB, LMDB, etc. are storage engines. A process uses these storage engines to write data to memory (note: the storage engine may support persistent, volatile, or both types of memory).

A database management system (DBMS) is a higher level concept that often has multiple processes either on a single server or distributed across multiple servers. Simply stated, each process within a DBMS uses the storage engine to read/write to/from memory (persistent or volatile).

It's important to note that storage engines are a specialized area and require different skills from writing a DBMS. It's a big deal when other people write high quality storage engines because it makes it a lot easier to write a DBMS.

akbar501

vor 9 Jahrendiscuss

In my experience, your points about communication and onsite visits are key. Also, communication should be centralized and managed on both sides so there is not only a communication channel, but also clear lines of responsibility. Of course, communication is not just phone and email, but also well defined specs.

However, getting involved in another company's hiring process is rarely a good idea. Most offshore projects are results oriented, so getting involved in hiring is too low level (i.e. micro-management). In other words, manage the output, not the process.

With the above said, involvement in hiring can be effective when 1.) you're using the offshore firm to acquire very specialized, high-end talent, and 2.) you're paying a substantial premium above local market to ensure hiring is a rare event.

akbar501

vor 10 Jahrendiscuss

Steve Forbes wrote about the lack of planning and impact on ordinary citizens: http://www.forbes.com/sites/steveforbes/2016/12/22/what-indi...

akbar501

vor 10 Jahrendiscuss

There is simply no factual basis for this comment.

Every American Muslim I have met believes in the 1st Amendment.

Perhaps more than many Americans, American Muslims are acutely aware of how our laws keep them safe. Reducing these laws would make them more of a target then they already are.

akbar501

vor 10 Jahrendiscuss

In case anyone who is not a Java dev wonders, "what is a fat jar"?

A fat jar is a single runnable binary. It makes deploying Java super simple and is roughly equivalent to Go's single binary.

akbar501

vor 10 Jahrendiscuss

AWS becomes interesting when you want to automate your infrastructure (i.e. infrastructure as software). When you're not orchestrating your infrastructure, then its probably overkill.

DO, as you rightly mentioned, is a lot closer to a straight forward Linux or whatever instance (easier to use). And the prices are very attractive.

I use both and split based on the above. DO is great for hosting web apps / dynamic web sites at a reasonable price. AWS is great for orchestrated clusters / large scale production, or for very low cost static hosting via S3.

akbar501

vor 10 Jahrendiscuss

Compose.io is owned by IBM. I had a chance to meet both the Compose team and a few people from IBM at the DataLayer conference. They are passionate and smart.

Compose is in a sweet spot right now. They run with the speed of a startup, but have access to IBM's massive resources. Really the best of both worlds.

akbar501

vor 10 Jahrendiscuss

It's the full Redis API and protocol. See https://github.com/Netflix/dynomite/blob/dev/notes/redis.md for a list of all supported Redis features.

The benefit of Dynomite's support for the Redis API + protocol is a.) you can use any Redis client and b.) the same code can be used for standalone Redis on your laptop and on a distributed Dynomite cluster.

akbar501

vor 10 Jahrendiscuss

Yes, I was a speaker at DataLayer.

There are two answers to your first question.

1. Dynomite pairs well with Cassandra. At scale, Cassandra is not a speed deamon when it comes to reads. Dynomite helps to improve Cassandra's read performance.

2. The use case for Dynomite as the primary database is for workloads that require high throughput and low latency. In other words, Dynomite delivers consistently lower latency at any scale.

akbar501

vor 10 Jahrendiscuss

https://github.com/Netflix/dynomite

Dynomite for production proven, limitless scale for Redis.

Provides ability to Redis for both in-memory and on-disk workloads.

Dynamo-inspired linearly scalable, shared nothing multi-master architecture. Pairs well with Cassandra.

Supports pluggable backends (ex. RocksDB).

Been in production 2+ years. One of the larger clusters handles over 3.6 million sustained ops/sec in production, every day.

Submissions

Ask HN: What's the state of the job market in data engineering?

comments