GOAT (Generative Offensive Agent Tester, arXiv:2410.01606): an attacker LLM
holds an in-context toolbox of 7 jailbreak techniques and reasons in an
Observation/Thought/Strategy/Reply structure each turn, sending only the Reply
to the target. Now available in an evals SDK for any agent harness SDK.
Achille and Soatto (Amazon/AWS) model AI reasoning agents as compute-capable stochastic dynamical systems and ask: can chain-of-thought reasoning solve any computable task? They show the key resource is inference time and that training data can exponentially reduce the time needed.
The initial version on GitHub does not implement caching or memorization but it's possible and where the project will likely head. (FYI I'm on the Strands Agents team).
They're handy for situations where it would be impractical to anticipate the way your input might vary. Like say you want to accept invoices or receipts in a variety of file formats where the data structure varies but you can rely on the LLM to parse and organize. AI Functions lets you describe how that logic should be generated on-demand for the input received, with post-conditions (another Python function the dev write) which define what successful outcomes look like. Morgan wrote about the receipt parser scenario here: https://dev.to/morganwilliscloud/the-python-function-that-im...
(FYI I'm on the Strands Agents team)
I'm excited to see what sorts of CloudFormation templates people have lying around or which previously shared templates people find interesting.
To continue the show and tell part of Show HN, here are a few examples that caught my eye.
This example architecture that meets the Australian Cyber Security Centre’s Information Security Manual for the Protected classification level. It includes patterns you can borrow for three types of private subnets, AWS KMS-based encryption, and AWS WAF firewall protection.
https://www.stackery.io/registry/?repo=https://github.com/aw...
Maybe you want to go back in time to 2019 when our problems involved how to construct serverless data lakes. If so, AWS Samples has you covered with this example which helps you think about both data ingestion and storage across several services.
https://www.stackery.io/registry/?repo=https://github.com/aw...
Someone shared the DataDog template for deploying their integration, which you should scope the IAM Role to your resources before you deploy it, but it’s wonderful to see vendors providing CloudFormation templates for their stuff.
https://www.stackery.io/registry/?repo=https://github.com/Da...
Finally, if you’re feeling nostalgic for winter, this example from Symphonia is a lab for benchmarking Lambda cold starts for Java runtimes so that you can better understand how they behave and plan configurations accordingly.
https://www.stackery.io/registry/?repo=https://github.com/sy...
Developer experience for serverless is such a pain point, spot on. AWS SAM has tackled some of the IaC modeling problem (on top of CloudFormation which is a mature choice) and they've had a crack at the local iteration (invoke Lambda function or API Gateway locally with connectivity to cloud services).
It's a little incomplete, missing some of the AWS IAM automation that makes local development smooth, environment management for testing and promoting changes, and some sort of visualization to make architecture easier to design as a team.
I work for a platform company called Stackery which aims to provide an end-to-end workflow and DX for serverless & CloudFormation. Thanks for comments like these that help identify pain points that need attention.