⛄ The workout regime transformed him into the abdominal snowman
Late to the Party 🎉 is about insights into real-world AI without the hype.
Hello internet,
I finally got around to building the new domain for Late to the Party 🎉!
Sorry for breaking your email filters. I’ll be sending the newsletter from this address now. If you’re a long-time reader, please do me a favour and add hello@late.email to your contacts!
I wrote an entire page that makes sure this newsletter actually hits your inbox!
Anyway, we have some goodies today! Let’s dive into some machine learning!
The Latest Fashion
- With TorchExplorer, you can interactively inspect your model, gradients and more
- Deep learning helped discover the first “new” antibiotic after 60 years
- The UK Supreme Court ruled AI cannot be named the inventor
Worried these links might be sponsored? Fret no more. They’re all organic, as per my ethics.
My Current Obsession
The big thing is probably that this newsletter now has its own website and domain! 🎉
I wrote the page from scratch, and I think it doesn’t look terrible.
I’ve been working hard to get this published, and everything set up and migrated. Email stuff is complicated, so I hope this all went smoothly. If you want to ensure that this newsletter still reaches you, I made a page with Tips and Tricks you can do!
https://late.email
I’m back to work this week! Lots of fun things are happening, like Google publishing a new weather paper on Christmas Day.
Personally, I have started my 30-day challenge of exercising every day. So far, I have been holding up really well! I went bouldering twice, touched the weights, and did bodyweight workouts daily. I also created a bingo card for achievements and experiences I want to do in 2024. This feels like a super fun way to motivate myself to do some harder and lighter tasks and remember to also enjoy the process!
Thing I Like
I am planning a snow vacation, and the ski pants came just in time for the temperatures to hit -10°C (14F). So clutch!
They kept me warm on my bike even!
Hot off the Press
I wrote a piece about preparing your custom GPTs for the GPT store that was just announced.
I also wrote about my year 2023 in AI, which turned out to be quite fun!
In Case You Missed It
My ml.recipes got a bit of a facelift!
On Socials
I shared “Understanding Deep Learning” with Linkedin, and it was pretty popular!
Python Deadlines
We have two deadlines coming up, namely PyTexas and PyCon Namibia!
I have added PyCon Estonia to the upcoming CfPs; love it when a conference knows its CfP closing date this far in advance!
Machine Learning Insights
Last week I asked, How do you ensure the robustness of machine learning models against adversarial attacks?, and here’s the gist of it:
Ensuring the robustness of machine learning models against adversarial attacks is crucial, especially as these models are increasingly deployed in real-world applications. Adversarial attacks involve subtly modified inputs designed to deceive machine learning models into making incorrect predictions or classifications.
Abdul Khaleq Al-Qasaily on LinkedIn wrote:
Ensuring model robustness involves thorough adversarial testing, continual monitoring for anomalies, and incorporating defence mechanisms like adversarial training.
That covers a lot of it, so let’s go into depth on key strategies to enhance model robustness:
- Adversarial Training: This involves incorporating adversarial examples into the training process. Training the model on a mixture of regular and adversarial data teaches it to recognize and correctly classify these manipulated inputs. For instance, in meteorological applications, models predicting weather patterns could be trained with real and synthetically modified weather data to ensure they aren’t misled by abnormal or manipulated inputs. You can read more in this tutorial.
- Data Augmentation: Augmenting training data with various transformations (like noise addition, cropping, and rotating) can improve the model’s generalization ability and make it less sensitive to small perturbations in the input data. In Earth science, models for classifying satellite images can be trained with augmented images that include various atmospheric conditions or angles.
- Regularization Techniques: Techniques like dropout, L1/L2 regularization, and batch normalization can prevent overfitting and help the model generalize better to new data, including adversarial examples. One can even combine this in adversarial training to increase regularization on vulnerable data samples!
- Model Architecture Choices: Certain architectures may inherently be more robust to adversarial attacks. For example, models with higher capacity (more layers/neurons) might resist small perturbations better. However, this also depends on the specific task and data. Different choices in model architectures were investigated and published by high-school students! (How cool is that?!)
- Defensive Distillation: This technique trains a model to output softened probabilities rather than hard classifications, making it harder for an attacker to craft effective adversarial examples.
- Detection Mechanisms: Implementing systems that detect and flag potential adversarial attacks can be an effective line of defence. These could be separate models specifically trained to recognize adversarial inputs. We must evaluate how new data samples fit into the training data distribution. Or even uncertainty estimates, which can help detect adversarial samples in addition to general model robustness.
- Benchmarking and Continuous Evaluation: Regularly testing the model against known adversarial attack methods helps understand its vulnerabilities. Continuous evaluation, similar to the detection mechanisms, ensures that the model adapts to new types of attacks or changes in data patterns.
- Collaborative Approaches: Sharing knowledge and techniques within the machine learning community, especially regarding new types of adversarial attacks and defence strategies, can significantly enhance the robustness of models across different applications.
- Offensive Security: Step-by-step operational machine learning models are exposed to the public internet. This exposure necessitates testing these models offensively, “trying to crack them”, just like we do with security-critical applications in other aspects of computation.
In the context of Earth science and meteorology, these approaches can be specifically tailored to ensure that models used for climate prediction, weather forecasting, and environmental monitoring remain accurate and reliable even when faced with potential adversarial manipulations. These may, however, be accidental due to instrument drift rather than malicious intent. Nevertheless, it is crucial to maintain accuracy to retain trust in these systems, especially given their significant impact on decision-making in areas like disaster management and agricultural planning.
Got this from a friend? Subscribe here!
Data Stories
The last months in Germany have been marked by flooding in many parts.
When visiting my friends in the North, they mentioned that one single municipality in the state is not one category below the emergency state.
Looking at the global monthly temperature anomaly, this makes sense.
These have been the warmest months on record at the end of the year.
Terrifying honestly.
Source: Copernicus
Question of the Week
- What are the challenges and solutions in handling time-series data in machine learning?
Post them on Mastodon and Tag me. I’d love to see what you come up with. Then, I can include them in the next issue!
Tidbits from the Web
- Is liking your job bad for your career?
- The 7 habits of highly miserable people
- A new Fantasy High season started Wednesday. Here’s a recap of the last two years!
This is a little personal story for those who are still reading. I was chilling in my bed on Tiktok in December, struggling with the long dark hours. When the Fantasy High: Junior Year trailer popped up, it gave me something to look forward to in the new year. Can’t wait! I’ll watch the first episode after I hit send on this! Have a great 2024!
Jesper Dramsch is the creator of PythonDeadlin.es, ML.recipes, data-science-gui.de and the Latent Space Community.
I laid out my ethics including my stance on sponsorships, in case you're interested!