🌋 Volcanoes lava good time
In this issue, we have LLMs from scratch, the harm of ChatGPT, a cool profiler for your Python scripts, an update from the Saima family, lots of science communication, and open-source weather deep learning! I finally answer the question of how AI can aide in volcanic eruption warnings. And, of course, we close out with some TikToks from last month!
Late to the Party 🎉 is about insights into real-world AI without the hype.
Hello internet,
ahh, it’s been a while! I apologise. Finally, I have internet again! So, if anyone was wondering if I pre-write these or have an AI do it, that should settle it.
In this issue, we have LLMs from scratch, the harm of ChatGPT, a cool profiler for your Python scripts, an update from the Saima family, lots of science communication, and open-source weather deep learning! I finally answer the question of how AI can aide in volcanic eruption warnings. And, of course, we close out with some TikToks from last month!
Let’s dive into learning about LLMs first!
The Latest Fashion
- Build a Large language model from scratch!
- How patriarchal AI can harm women’s careers
- Pyheat is a profiler that plots, which line of code you’re spending time on
Worried these links might be sponsored? Fret no more. They’re all organic, as per my ethics.
My Current Obsession
Oh my, it’s been so long! I finally have internet access at the new place! My move went well. Everything is roughly where it belongs, and nothing is broken. Now, the eternal fight against boxes begins. This is, in fact, not helped by me having internet and being able to ignore the boxes. But generally, I am so happy I moved already. My old place was a bit away from the city centre and my commute to work was 40 minutes by bike and over an hour by public transport. Mind you, Bonn is not that big, I just really lived at the least optimal location. Now, my commute is 20 minutes down the Rhine by bike, which is so beautiful! Also, in two and a half years out there, I met people three times. In 24 hours, I have beaten that number in the new place. I feel much more like I live in a community, and it’s beautiful.
Speaking of community, the Saima family managed to evacuate the kids and mother. Thanks to your and everyone else's support! There’s still a way to go to get the father out, so if you have $5 to spare, consider helping someone in war-torn Gaza through Project Olive Branch.
If you live in Europe, remember to vote in the European elections. Germany votes today, so if you’re in Germany and don’t know who to vote for, check out the Wahl-O-Mat, which lets you answer questions and shows which party is aligned with your values. (Such a neat idea, right?!)
Last time, I also mentioned my talk at the Bonn Science Night, and it went really well! Lots of people and engagement, I really appreciated that open communication, and it rekindled my love for sci-comm. We’ll see if I can nurse that flame back into something that was before.
On another note, I was featured on the ECMWF website in a staff profile. Such an honour! The Director General of ECMWF, Florence Rabier, said some really heart-warming things, too. I will not disclose whether I cried a few happy tears or not.
Finally, in a push to open source what we are learning from the development of our AI-aided data-driven weather forecasting system AIFS at ECMWF, we have started to create the Anemoi ecosystem. Large parts of the inference pipeline, including the core model, are open already. My colleagues and I have spent an incredible amount of effort to push this and also make it publicly available. You can check out the Anemoi Models documentation! I really hope this will be useful to accelerate the development of new models and use cases in the weather community.
Thing I Like
This is a bit symbolic, but being back behind my nice mechanical keyboard, able to type on a normal screen is just lovely. I missed having a full-fledged computer.
Hot off the Press
Python Deadlines
This is a long list, so if you'd rather see them yourself, just visit: PythonDeadlin.es
I added a few new CfPs in the last month. Starting with PyCon Africa, PyCon Australia, PyCon Latin America, PyCon Netherlands, PyCon Niger, PyCon Poland, PyCon South Korea, PyCon Zimbabwe, EuroScipy, PyBay, PyHEP, PyTexas, PyDay CopiapĂł, PyDay Valparaiso, Django Day CPH that have CfPs. I also found the dates for PyCon Sweden and PyCon Ireland. For 2025, you can mark the dates of PyCon Italia, PyCon US, and PyCascades.
The CfPs for PyCon Latin America and EuroScipy are ending soon.
Machine Learning Insights
Last issue, I asked, Can AI assist in more accurate and timely prediction of volcanic eruptions?, and here’s the gist of it:
Machine learning systems can process and integrate vast amounts of data from various sources, such as seismic activity, ground deformation, gas emissions, thermal imagery, and historical eruption patterns. Machine learning algorithms can analyze this data to identify patterns and correlations that might not be directly obvious from classic analysis.
1. Seismic Activity Monitoring
Volcanoes sometimes emit seismic signals before an eruption. AI models, particularly those using deep learning, can be trained to recognize the subtle changes in seismic patterns that precede an eruption. These models can analyze continuous streams of seismic data to provide early warnings. I actually participated in a Kaggle challenge at Mt. Etna in Sicily for this.
Back in the day, I remember some volcanic columns actually having a resonant frequency in the magma column, similar to a flute. This frequency would change when more magma entered the column, or pressure changed. This doesn’t happen everywhere, so I doubt we’d get a volcanic foundational model anytime soon, but still!
2. Ground Deformation Detection
Satellites and ground-based sensors monitor the deformation of a volcano’s surface, which can indicate magma movement or calderas. AI algorithms can process InSAR (Interferometric Synthetic Aperture Radar) data to detect even small ground movements, providing crucial information about potential eruptions.
3. Gas Emissions Analysis
Volcanoes release various gases, such as sulfur dioxide, before an eruption. Machine learning models can analyze data from gas sensors to identify changes in emission levels and composition, which can signal an impending eruption. A classic algorithm would be to compare long-time averages to short-time averages, but these measurements are both hard to do and hard to automate. Technically, one could use change-point detection algorithms or recurrent neural networks, but I think due to the data availability, I haven’t been able to find much research on this.
4. Thermal Imaging
Back to something more easy! Thermal cameras and satellite imagery can detect changes in temperature around a volcano. AI can analyze these thermal images to identify hot spots and anomalies that might indicate volcanic activity or other anomalous patterns.
Closing Thoughts
It’s incredibly important to validate these models correctly, as sending wrong alerts can be as harmful as not sending out alerts when an eruption is imminent. But when implemented correctly, they can be used to inform the correct authorities to trigger an alert and save lives in the community.
Got this from a friend? Subscribe here!
Question of the Week
- How do you address the challenge of data scarcity when applying ML to rare environmental phenomena?
Post them on Mastodon and Tag me. I'd love to see what you come up with. Then, I can include them in the next issue!
Tidbits from the Web
- I love this “movie pitch” and the punny title!
- This pregnant creator has been the target of a harassment campaign.
- Provided without context.
Jesper Dramsch is the creator of PythonDeadlin.es, ML.recipes, data-science-gui.de and the Latent Space Community.
I laid out my ethics including my stance on sponsorships, in case you're interested!