🦌 Oh, deer. Where's the time?!
In this issue, we have problematic AI search, weather models in a changing climate, and fractals! I talk about burnout, being creative, and I dig deep into reinforcement learning. There’s also a scuba picture!
Late to the Party 🎉 is about insights into real-world AI without the hype.
Hello internet,
That is not dead which can eternal lie… I’m back! I hope you have a lovely season of lights, my favourite part about winter. Making it nice and cozy!
In this issue, we have problematic AI search, weather models in a changing climate, and fractals! I talk about burnout, being creative, and I dig deep into reinforcement learning. There’s also a scuba picture!
Let’s dive right in!
The Latest Fashion
- Turns out Google, Microsoft and Perplexity promote scientific racism in AI results
- How do AI weather models perform in a changing climate?
- Are fractals the answer to scaling computer vision models?
Worried these links might be sponsored? Fret no more. They’re all organic, as per my ethics.
My Current Obsession
Well… it happened. I fully burnt out. This year has been really rough on me. I wasn’t able to keep up most of my self-care habits, despite noticing and trying again and again. I don’t really know what is happening, and it feels impossible to reduce what I do even more. After all I’m barely keeping up with my household, as well as work. If you know some good resources, let me know. I’m interested.
Unfortunately, I wasn’t able to participate in the Advent of Code this year either. I just don’t have the energy. I wish I could, as I usually enjoy it every year. But realistically, I also get way in over my head on AoC, so it’s better to be careful. Let me know if you’re partaking this time! I’d love to see your Github repo.
That being said, I’m trying to have more fun with things. So I am trying out Ali Abdaal’s Part-Time Youtube Academy. I hope this will be a path towards creating more original and interesting videos after all that time of struggling with it. I hope I can make this a fun hobby, because realistically, I don’t depend on it with my full-time job. Wish me luck!
On a positive note, I went on a big 2 week trip to Egypt to do a lot of diving in the Red Sea and to visit the Valley of Kings and the Temple of Karnak in Luxor. I dove 18 times. Had sharks circle my feet. Got up close with a manta ray. Saw the “Spanish Dancer” sea slug (it’s SO red!). And had a lot of fun under water. Highly recommend the Serenity liveaboard if you’re considering a trip! Here’s me not noticing a shark right above me, until we checked the photos.
Thing I Like
I’ve been going back to hand-written notes now to take the speed out of life a bit.
Hot off the Press
In Case You Missed It
My post about cached properties in Python is now more popular than my post about laptops for machine learning….
On Socials
I boosted a post about evaluating the robustness of data-driven weather forecasts in changing climates by my colleague Thomas Rackow, which got quite popular. I also had a lovely little chat on a post about the magic wormhole.
Python Deadlines
I found some new Python deadlines for PyCon Germany & PyData Conference and DjangoCon Europe.
We have upcoming deadlines for PyCon Germany & PyData Conference, PyCon US, PyCon Asia Pacific, PyCon Web.
Machine Learning Insights
Last week I asked, What are the practical applications and limitations of reinforcement learning in real-world scenarios?, and here’s the gist of it:
Reinforcement learning (RL) represents a significant methodological framework within machine learning, particularly noteworthy for its ability to learn through interaction with environments. I particularly love RL's iterative improvement process and how they refine predictions through continuous feedback loops.
Practical Applications
The application of RL in environmental control systems demonstrates particular promise:
- HVAC optimization in large buildings, where the system learns to balance energy efficiency with occupant comfort
- Smart grid management, adapting to varying renewable energy inputs and demand patterns
- Water treatment facility operations, optimizing chemical usage and processing times
The promise is that RL agents can effectively manage complex, interconnected systems with multiple competing objectives.
Critical Limitations
The Reality Gap
One of the most significant challenges I've encountered in playing around with RL systems relates to what we term "the reality gap" - the disparity between simulation and real-world performance.
Key Challenges:
- Real-world environments are inherently noisy and unpredictable
- State spaces are often continuous and high-dimensional
- Actions have delayed and compound effects
- Safety constraints must be rigorously maintained
Data and Computational Requirements
The computational demands of RL systems present substantial practical barriers:
- Training requires extensive interaction with the environment
- Real-world training can be prohibitively expensive or dangerous
- Simulation environments may not capture critical real-world complexities
And it cannot be understated that setting up RL experiments is often really difficult.
Methodological Considerations
Safety and Reliability
Drawing from my experience with weather prediction systems, I've observed that ensuring safe exploration during learning represents a fundamental challenge:
- Critical systems cannot tolerate significant failures during learning
- Safety constraints must be incorporated into the reward structure
- Verification and validation of learned policies remain challenging
Scalability Issues
The scalability of RL solutions presents particular challenges:
- State and action spaces grow exponentially with problem complexity
- Transfer learning between different environments remains limited
- Resource requirements often scale non-linearly with problem size
Future Directions and Research Opportunities
Recent methodological advances suggest promising directions:
- Hybrid approaches combining RL with traditional control methods
- Improved simulation environments for training
- Better transfer learning techniques to reduce real-world training requirements
Conclusion
While reinforcement learning shows remarkable promise in various domains, its practical implementation faces significant challenges. The field requires continued research to address these limitations, particularly in areas of safety, scalability, and real-world applicability.
Understanding these limitations proves crucial for practitioners. Just as functioning weather models must account for uncertainty and chaos in their predictions, RL systems must similarly acknowledge and adapt to the inherent complexities of real-world applications.
Got this from a friend? Subscribe here!
Question of the Week
- How can AI be used to improve the efficiency of renewable energy systems like wind turbines or solar panels?
Post them on Mastodon and Tag me. I'd love to see what you come up with. Then I can include them in the next issue!
Tidbits from the Web
- Folding paper can be very meditative, why not try 25 days of folding fun?
- The “Bike Bus” is a neat concept to get kids to school without an onslaught of SUVs.
- I’m late for this one, sorry, but Josh Sundquist had another amazing Halloween costume!
Jesper Dramsch is the creator of PythonDeadlin.es, ML.recipes, data-science-gui.de and the Latent Space Community.
I laid out my ethics including my stance on sponsorships, in case you're interested!