GFlowNets for Signal Miner: A New Way to Find Diverse, High-Performing Models

Data Science
1

Hey everyone,

I wanted to share something exciting I’ve been working on that could significantly improve the search for diverse, high-performing models in the Numerai ecosystem. If you’ve been using Signal Miner (GitHub), you already know the key challenge: predictive edge is small and fleeting. Strong models might hit 5% correlation, maybe 10% at best, and poor performance (even negative correlation) is inevitable in certain rounds.

But one thing we do know: multiple diverse competing solutions can exist and score well. Having many well-scoring models is better than having just the single best one.

GFlowNets: A New Paradigm for Model Search

Last week, I attended a talk by Alex Hernandez-Garcia introducing GFlowNets, a fascinating idea championed by Yoshua and Emmanuel Bengio. If you haven’t heard about them, check out:

Quote from Dr. Bengio:

“I have rarely been as enthusiastic about a new research direction. We call them GFlowNets, for Generative Flow Networks. They live somewhere at the intersection of reinforcement learning, deep generative models, and energy-based probabilistic modeling. They are also related to variational models and inference, and I believe they open new doors for non-parametric Bayesian modeling, generative active learning, and unsupervised or self-supervised learning of abstract representations to disentangle both the explanatory causal factors and the mechanisms that relate them.”

What makes GFlowNets special?

Unlike traditional deterministic optimization, GFlowNets are generative models trained to produce diverse outputs that all perform well on a given task. Instead of just finding the best single solution, they learn a probability distribution over good solutions. This is incredibly useful for problems like drug discovery, where you need to explore many promising molecules rather than just one.

Applying GFlowNets to Signal Miner

This approach is exactly what we need for Signal Miner. Right now, Signal Miner samples hyperparameters uniformly from LightGBM. But what if, instead of uniform sampling, we trained a GFlowNet to generate hyperparameter sets that consistently produce high correlation with the Numerai target?

From the NeurIPS paper:

“This paper is about the problem of learning a stochastic policy for generating an object (like a molecular graph) from a sequence of actions, such that the probability of generating an object is proportional to a given positive reward for that object.”

GFlowNets allow us to:

  • Find a diverse set of benchmark-beating models (instead of just one optimal set of hyperparameters).
  • Adapt dynamically to what works best, rather than relying on uniform random sampling.
  • Speed up hyperparameter search by focusing on promising regions instead of exhaustive grid search.

Implementation & First Results

I built a Signal Miner environment within the GFlowNet framework, available here:
:link: GitHub: GFlowNet Signal Miner

To train, just run:

python main.py env=signalminer proxy=signalminer logger.do.online=True

This streams training results to Weights & Biases.

Early Findings

I ran a lightweight experiment on a subset of Numerai Classic data and features. Each training batch evaluates and updates 10 sets of parameters. Over 12K rounds have processed so far. (Clarification: “reward” and “proxy” mean the same thing.)

The trend is clear: over time, the GFlowNet generates better hyperparameters that achieve higher mean reward. Interestingly, the max score has not increased significantly yet, which suggests further tuning is needed. But for an initial test, this is very promising!

Example of GFlowNet-Generated Hyperparameters

In GFlowNet terminology, model output is called a state, representing a sequence of actions that led to a final configuration. Here’s an example of what GFlowNets produce when searching for optimal hyperparameters:

Final state: [1, 5, 10, 2, 3, 8, 7, 9]
Sequence of actions: [(1,), (5,), (10,), (2,), (3,), (8,), (7,), (9,), (-1,)]
Human-readable version:
colsample_bytree: 0.34 | reg_lambda: 0.0 | learning_rate: 0.33334 | max_bin: 5 | max_depth: 9 | num_leaves: 18 | min_child_samples: 1000 | n_estimators: 23

Training Stats
Training Stats1920×405 58.7 KB

Why This Matters

This experiment is just the beginning. GFlowNets are generative AI beyond language models—they use probabilistic reasoning to explore solution spaces efficiently. In our case, we want to generate a field of diverse, high-performing models, not just one single best model.

This aligns with principles of ensemble learning and diversification—critical to Numerai’s long-term success. It also aligns with the vision of Signal Miner: mine for your own unique alpha. Everyone is a winner!

Next Steps

The main challenge now is speed—we need to evaluate many models to properly train the GFlowNet. However, new research is showing how we can use cheaper (faster) low-fidelity proxies to help train GFlowNets more efficiently.

Check out Multi-Fidelity Active Learning with GFlowNets for more on this. Incorporating this approach could make training significantly faster and more cost-effective, allowing us to explore even more parameter spaces efficiently.

But I’m excited about the potential. This could be a powerful new tool for Numerai tournament participants.

Would love to hear your thoughts! Have you tried GFlowNets? Do you think this approach could be applied elsewhere in the tournament? Let’s discuss! :rocket:

2 Likes