Show HN: Multiple – Load test any API with JavaScript and NPM packages(app.multiple.dev)
app.multiple.dev
Show HN: Multiple – Load test any API with JavaScript and NPM packages
https://app.multiple.dev/
15 comments
One problem with products like this is that the same marketing materials are used by DDoS vendors. In their case of course "load testing" is coded language for "DDoSing a target." But nonetheless, you run the risk of people abusing your service to "load test" services they don't control, which could lead to high bills on your side that might never even get paid (people who DDoS others tend to have a lot of overlap with people who pay with stolen credit cards).
Do you have a plan to avoid this? I think it can be solved with simple domain verification. Generate a unique key for the customer and ask them to insert it into a DNS TXT record to prove they control the domain you're about to spam with requests on the behalf.
However, given the flexibility of your platform, that might not be sufficient. If a user can write arbitrary scripts, then can they change the target mid-test? You might also need some kind of firewalling solution in place.
Do you have a plan to avoid this? I think it can be solved with simple domain verification. Generate a unique key for the customer and ask them to insert it into a DNS TXT record to prove they control the domain you're about to spam with requests on the behalf.
However, given the flexibility of your platform, that might not be sufficient. If a user can write arbitrary scripts, then can they change the target mid-test? You might also need some kind of firewalling solution in place.
This is a topic we think about a lot. The downside to requiring domain verification is additional friction, and if an external API call is required (e.g., logging in via Auth0), those tests may not be possible.
Accounts on the free tier won't be able to create much of an attack. We perform customer verification on accounts with higher concurrent virtual user limits.
We closely monitor potential abuse and will adjust our verification strategy as needed. Thanks for the question.
Accounts on the free tier won't be able to create much of an attack. We perform customer verification on accounts with higher concurrent virtual user limits.
We closely monitor potential abuse and will adjust our verification strategy as needed. Thanks for the question.
Looks neat! What's the warm-up time on tests? Any maximums on concurrent users this can simulate? (does warm-up time increase with # concurrent users?)
Thanks!
> What's the warm-up time on tests?
Tests usually take 20-40 seconds to spin up. Tests with over 10k virtual may take up to 60 seconds to start.
> Any maximums on concurrent users this can simulate?
The free tier supports up to 50 virtual users. Paid tiers can go up to 50k virtual user
> What's the warm-up time on tests?
Tests usually take 20-40 seconds to spin up. Tests with over 10k virtual may take up to 60 seconds to start.
> Any maximums on concurrent users this can simulate?
The free tier supports up to 50 virtual users. Paid tiers can go up to 50k virtual user
50 VUs on the free plan is what I understood
Congrats! Excited to play with this. For anyone else looking for the docs I found them through the landing page (the top level post direct links to the app): https://docs.multiple.dev/quick-start-guide
Thanks for the feedback and sharing the link
I tried Multiple's service out in an early beta preview, and was impressed with the flexibility of the initial product. Customer service and support was very good as well. The current product roadmap will be adding a lot of compelling features in the coming months.
I'd absolutely recommend anyone who needs load testing tooling and infrastructure to evaluate this service, and I am excited to see where they go.
I'd absolutely recommend anyone who needs load testing tooling and infrastructure to evaluate this service, and I am excited to see where they go.
Great product! I really liked the UI/UX, and the fact that you provide an example is awesome. I played around with it for about 10 minutes, but one thing wasn't entirely clear to me: How do I import my API if it's not an NPM package?
Thanks for the feedback! Today, you need to script API calls with an http client like axios. We're working on a feature that will let you import a swagger or har file, and autogenerate most of the test.
Haven't played with the product itself, but I have known Yev for some time now. Sure he and the team will do something meaningful in this space. All the best.
Big fan of Multiple! You should give your designer an extra Hi5
You mentioned CLI. When is that coming and what will it do?
The CLI is slated for end of Q4/early Q1. It will be able to run load tests, get results, manage your team, and most of the other functionality you get in our web app.
One major update included with the CLI is a code repo integration - you can have your load test scripts in your codebase and track results to commits.
One major update included with the CLI is a code repo integration - you can have your load test scripts in your codebase and track results to commits.
Awesome! Love how JS forward it is.
I wanted a better load testing solution – so I built one with my team at Multiple. We just opened early access and would love to get your feedback.
We created Multiple to solve three challenges with existing tools:
1. Limited scripting capabilities. XML or GUI-based scripting can only test basic scenarios. Existing code-based tools struggle with auth, generating synthetic data, and testing anything other than HTTP requests. We went with JavaScript for ease of use, versatility, and integration with existing developer workflows.
2. Cannot use existing libraries or code. Instead of forcing you to learn a new system and rewrite code, Multiple leverages the JavaScript and NPM ecosystem so you can use packages you're already familiar with. By supporting NPM packages, Multiple can test nearly any API, service, or protocol.
3. Tedious infrastructure management. There's no reason to spend time spinning up and configuring machines, and then destroying them after a test. Multiple abstracts that away. You just enter the test size and duration, and press start.
My favorite feature we've built so far is the Debug Run. You can use Debug Run as you write your tests to execute a single run-through. It's helpful to verify correct behavior and capture logs, and it allows you to iterate quickly, without spinning up a full load test each time.
We have so much in store for developers: pass/fail conditions, CLI, and repo integration, to name a few. Thanks for reading, and let us know what you think.