🎄 Lucky I got an advent calendar, their days are numbered
Late to the Party 🎉 is about insights into real-world AI without the hype.
Hello internet,
The Christmas season started! Let’s look at some merry machine learning!
The Latest Fashion
- Love this idea! They published a high-quality audiobook version of the Stochastic Parrot Paper!
- The Incredible Pytorch is a curated list of various content around Pytorch
- Friends don’t let friends make pie charts.
Got this from a friend? Subscribe here!
My Current Obsession
It’s that time of the year again! 🎄 Advent of Code 🎄 is on!
This was probably the most challenging Day 1 ever in the history of AoC, but I’m once again in love with code puzzling every day. In case you don’t know it yet, it’s an advent calendar, where you unlock a puzzle with two parts every day. They’re usually more accessible in the first week and more complex on the weekends when people have more time to spend on these. The creator of these puzzles tries to word them specific enough to make a solution possible but ambiguous enough for different solutions to work.
One year ago, I built a custom queue solution for a problem. Some people used async calls since they’re web developers, and some folks used multi-processing to build a thread pool. For the same text, just have different numbers in them. It can be truly versatile.
I’ve been spending all week at ECMWF finishing up a multi-1000 line pull request to make our models ready for experimentation by scientists! My brain is smoking!
If you’re on TikTok, sometimes the app just wants you to watch a series. I started watching The Lincoln Lawyer, and it’s pretty entertaining so far.
Thing I Like
I finally finished setting up and decorating my Christmas tree. Super happy to have a sparkler in my house for the dark season!
Hot off the Press
In Case You Missed It
My post about choosing a laptop for machine learning is quite popular again!
On Socials
I posted about the Advent of Code and my plans this year. Both Linkedin and Mastodon really loved it!
Linkedin also really liked MatPlotX, which makes Python plotting much easier, which I shared with you eight months ago! If you like things that make matplotlib easier, you may like my short video about Proplot!
Python Deadlines
We have a new Python conference for PyCon Namibia next year!
Machine Learning Insights
Last week I asked, What revolutions in AI do you expect in 2024?, and here’s the gist of it:
It seems this question is on the minds of many. Ian Ozswald, who organises Pydata London and writes over at Not A Number (highly recommended), phrased this very interestingly:
Thinking of Generative AI within the next 1 year, suggest something Possible, something Plausible and something Probable.
As my brain works, I immediately gravitated to some damning predictions, so I extended it!
One positive and one negative.
Probable
Positive: Code tools like Copilot progress to the point where they can take over most of the “grunt work” on the limited generative task of “code logic”. The hard part was always conceptualising the system that the code is running in, and code chatbots (like Copilot Chat in beta right now) will increase the code quality and make documentation a breeze.
Negative:Â The Spampocalypse. Believable-looking and sounding emails and Reddit posts are nothing. Videos with cloned influencers promoting trash. Content mills churn out whatever trite trash they think gets clicks. This goes hand in hand with more believable phishing attacks.
Plausible
Positive: Instead of these massive data-hungry LLMs and distillation, we’ll get a new model with limited reasoning capability. Could be Bayesian, more like some sort of causal inference that points X -> Y and is hailed as the solution for hallucinations. It won’t be, but it will make for more efficient training paradigms.
Negative: Legislation will fall to the siren call of longtermism and regulate “a possible future AGI”, listening to the main hypemen in private AI development, which will negatively impact the current real-world impact genAI has on marginalised communities (be it environment or direct through biases).
Possible
Positive: A generative model will run on robotics hardware and create something that is aesthetically mind-blowing. There will probably be hacks involved. Like OpenAI showing the Rubik’s cube solving hand, which wasn’t taught from scratch. But it will bring generative AI into the physical world.
Negative: We had the enshittification of social media. So it’s entirely possible that these impressive but expensive models get hamstrung and monetised. Capitalism might get us all, and Google possibly has people asking the question, “How can we sell ads in a Bard generative output?”. The slow-marching grip of capitalism will get us all in the end.
But I kinda wanted to add some bonus predictions:
- The EU will force tech companies to invest in unlearning tech so that you can tell Meta to forget your data. Tech will basically exclude you from their services if you do.
- A bunch of small firms will push into genAI, just like into data science, without even having a use case.
- Coincidentally, a bunch of companies will have their secrets leaked due to a lack of LLM security.
- Even the most sophisticated models behind a paywall still have to do significant postprocessing to get faces, text, and fingers right 100% of the time.
But those are basically just one step ahead, so easy predictions!
Question of the Week
- What Christmas gift are you getting the machine learning scientist in your family?
Post them on Mastodon and Tag me. I’d love to see what you come up with. Then I can include them in the next issue!
Tidbits from the Web
- Oh my, the Dunning-Kruger-Effect is just an artefact from autocorrelation…
- What happens if Microsoft throws their data centres into the ocean
- John Oliver’s segment on Dollar Stores was enlightening
Jesper Dramsch is the creator of PythonDeadlin.es, ML.recipes, data-science-gui.de and the Latent Space Community.
I laid out my ethics including my stance on sponsorships, in case you're interested!