Hi dang, sorry to ping you here again. I emailed back about a week ago but haven't heard back. My posts are still showing up as [dead] immediately after submission even with non-dev.to URLs. Would appreciate it if you could take a look when you get a chance. Thanks!
I built this because most RL environments are either interesting or accessible, rarely both. Engaging environments like MineRL or Dota 2 either demand massive compute or involve painful setup—dependency hell, rendering configs, CUDA mismatches, OOM errors.
Agenlus runs entirely in your browser using WebGPU. No install, no setup required. We start with redesigned classic control problems—CartPole, MountainCar, and their variants—with more environments on the way.
This is an early release. Would love feedback on what environments or features you'd find most useful.
This model is built on Qwen2.5-Coder-1.5B-Instruct and has been fine-tuned exclusively on the official Roblox Luau corpus. It is designed to assist developers with code generation, completion, and understanding of Luau patterns commonly used in Roblox game development.
Key Features
Fine-tuned on quality-filtered Luau code from the official Roblox corpus
Optimized for Roblox-specific APIs and development patterns
Trained on TPU v5e-8 infrastructure
Includes interactive Colab notebook for immediate testing
Technical Details
The model uses Low-Rank Adaptation with target modules q_proj and v_proj, trained on filtered data (20-5000 character code samples) with a 90/10 train-validation split. Training was conducted with a batch size of 4, gradient accumulation of 32 steps, and a learning rate of 3e-5 using cosine annealing.
Access
Model: Interactive Demo: Available via Google Colab (link in model card)
The model is released under Apache 2.0 license and is ready for integration into development workflows.
Feedback and contributions from the community are welcome.
I've always felt that standard benchmarks focus too much on final accuracy, while the architectural choices that get us there are often treated like a dark art. We celebrate a new SOTA model, but rarely do we have a common language to discuss why a specific operation is efficient or how it contributes to the whole.
That's why I built "The Architect's Arena," a GWO-based benchmark that tries to score the "architectural intelligence" of neural network operations.
The Core Idea (How it Works):
Inspired by the "Window is Everything" paper, it breaks down any operation (like a Conv layer) into its fundamental components (Path, Shape, Weight) and calculates a theoretical "Operational Complexity" score. The final efficiency score is a function of this complexity and the model's actual performance.
What's special about this benchmark?
We explicitly separate the leaderboard into two parts:
1. Baseline Operations: Individual building blocks like StandardConv. The goal here is to invent new ops with high efficiency scores.
2. Reference Architectures: Complete models like ResNet. This isn't for score competition, but serves as a "performance target" to aim for. The challenge is to use your efficient new ops to build an architecture that can match ResNet's accuracy with lower total complexity.
This is an early version, and I would love to hear your feedback. What do you think of this approach? Are there any crucial baseline models or datasets you think should be added?
For this initial version, I kept the gating static to keep the model as simple as possible while validating the core idea. Making the gate dynamic based on the input is a great suggestion for the next step, and I agree it could lead to better performance. I really appreciate the feedback
I built FactorizedAttention - a new attention mechanism based on
the GWO framework. Instead of simple QK^T dot products, it uses
factorized quadratic forms to model higher-order token interactions.
Testing on GPT-2 small + LoRA fine-tuning:
Math reasoning: 3.4% PPL improvement
Competitive programming: 3.2%
Python code: 1.9%
The bigger gains on reasoning tasks suggest the approach helps
with complex relationships. Still early stage (only GPT-2 small),
but the results are encouraging. Happy to answer questions! Code + repro steps in the repo.
That's a fantastic question, and you've hit on a perfect example of the GWO framework in action.
The key difference is the level of abstraction: GWO is a general grammar to describe and design operations, while Mamba is a specific, highly-engineered model that can be described by that grammar.
In fact, as I mention in the paper, we can analyze Mamba using the (P, S, W) components:
Path (P): A structured state-space recurrence. This is a very sophisticated path designed to efficiently handle extremely long-range dependencies, unlike a simple sliding window or a dense global matrix.
Shape (S): It's causal and 1D. It processes information sequentially, respecting the nature of time-series or language data.
Weight (W): This is Mamba's superpower. The weights are highly dynamic and input-dependent, controlled by its selective state parameters. This creates an incredibly efficient, content-aware information bottleneck, allowing the model to decide what to remember and what to forget based on the context.
So, Mamba isn't a competitor to the GWO theory; it's a stellar example of it. It's a brilliant instance of "Structural Alignment" where the (P, S, W) configuration is perfectly tailored for the structure of sequential data.
Thanks for asking this, it's a great point for discussion.
Hi HN, author here.
For years, it bothered me that convolution (the king of vision) and matrix multiplication / self-attention (the engine of Transformers) were treated as completely separate, specialized tools. It felt like we were missing a more fundamental principle.
This paper is my attempt to find that principle. I introduce a framework called GWO (Generalized Windowed Operation) that describes any neural operation using just three simple, orthogonal components:
Path: Where to look
Shape: What form to look for
Weight: What to value
Using this "grammar", you can express both a standard convolution and self-attention, and see them as just different points in the same design space.
But the most surprising result came when I analyzed operational complexity. I ran an experiment where different models were forced to memorize a dataset (achieving ~100% training accuracy). The results were clear: complexity used for adaptive regularization (like in Deformable Convolutions, which dynamically change their receptive field) resulted in a dramatically smaller generalization gap than "brute-force" complexity (like in Self-Attention).
This suggests that how an operation uses its complexity is more important than how much it has.
I'm an independent researcher, so getting feedback from a community like this is invaluable. I'd love to hear your thoughts and critiques. Thanks for taking a look.
The paper is here: https://doi.org/10.5281/zenodo.17103133