31 December 2024
In this section I’m gonna talk about what I did in 2024. What a year it was! Learned a lot of stuff this year, not as much as I wished (mainly because of how limiting school is).
The year started with some non-sense work for the FIICode Web contest, I had to build a web app in a couple of months following their guidelines. It was pretty tiring and not that satisfying, but I won 300$ in the end, third place at the nationals. Afterward, in February, I participated in the Informatics Olympiad for Teams (IIOT) in Piatra-Neamt where my team won Bronze. Had a great time, it was also relaxing and fun, I did some networking and I got to meet an old friend!
In march, I qualified for the national phase (so top 90 out of ~1300 students) of the Romanian Informatics Olympiad, which is competitive programming. This meant a lot for me since I got to prepare straight for a month, while I was allowed to skip any classes at school. Went there to the nationals in April, in Bucharest at the Politehnica University of Bucharest. This was a long dream of mine, and I was very happy I finally managed to achieve it. In the end, I got a Bronze Medal, placing on the 46th place out of ~90 students in 10th grade. This also enables me to be admitted to any CS faculty in Romania, quite an honor.
Right after this, I went to Brasov to the Romanian CanSat and Rocketry Championship, where I qualified with my team to build a CanSat. I did the embedded programming part. It was nice and a different experience since we rented an apartment near the institute where the competition took place. An opportunity to mess with arduino, sensors and C. Following this, I went to Slobozia to the C# Olympiad (for Windows Forms apps). This was more of a fun experience since the stakes were low and we were staying at an abandoned resort, Amara. Also, a bit disappointed at the end since the other guys were super try-hard, I barely had time to finish the app during the 5 hours timeframe. I also got a Bronze Medal.
I broke the “bronze” curse at the Infomatrix international competition in Bucharest. It was organized by the Lumina Foundation at ICHB. Here I went with the same app I built for FiiCode, since I believe in recycling, and guess what. The app which got me third place in Romania, got me a Gold Medal at an international competition! Yup, so for this, besides the app, I also prepared a lot of posters for the pit. It was a learning experience, also an intercultural one.
The last competition of the year was Infoeducatia, where I won the first place last year. Turns out I brought too much attention and I got disqualified, on third place, at the local level. That is before the national phase, where I previously won. So with the exact same project, I got: third place in Romania, a gold medal internationally, and third place in my city. What a goofy world! The guy who went to the nationals instead of me got a bronze medal.
After the competitions, I had the summer vacation. I was hyper-focused on my deep learning grind. I wanted to get really knowledgeable in this field, so I started working. Did a lot of courses during the summer and some smaller projects to practice (I got a prize thanks to one of them from Nvidia Romania! met wonderful people there). I think this is the only learning goal I set last year that I accomplished, however I am happy with this. I have much to learn in ML and I am confident I am building a strong foundation. This learning grind continued up until the very last day of 2024 (and it will continue in 2025). What can I say, discipline is the key.
To take a pause from talking about myself, let’s recap some major events I cared about in 2024, mainly in tech.
One of the most important things in AI this year could arguably be the fact that Geoffrey Hinton and John Hopfield won the Nobel Prize in Physics. This was an amazing achievement. It showed validation to the world that deep learning is a real thing and one that works. These are heroes in this field and I feel inspired by their achievement to continue studying artificial intelligence, as I see it as a beautiful combination of mathematics, computer science, and physics. It’s also awesome, and funny, that researchers from Google Deepmind won the Nobel prize in Chemistry for Alphafold! I might get bad grades in chemistry, but computer scientists won the Nobel Prize there! :D
We had a lot of OpenAI drama this year, they could even make a series about it. I’m most hopeful about Ilya Sutskever starting Safe Superintelligence Inc. I respect him a lot and I am sure he is doing the right things. Can’t wait to get some first updates from his new research.
The ML wish of 2018: may all your local minima be global, your variance bounded, your labelled data plentiful, and your compute massive!
— Ilya Sutskever (@ilyasut) December 31, 2017
This year was also a banger year for Nvidia. They’re on a winning streak and I believe a lot in their mission. I think compute is the most valuable resource these days. The compute must flow.
Notable LLM releases which I care about were Mistral Nemo 12B, Deepseek v3, more from Qwen, Pixtral, LLaMa. I like Mistral’s open source models a lot. I also like we have some AI player from Europe. As fair as I’m aware they’re the only big lab who released models using Mamba. I care the most only about open source LLMs which I can run on consumer hardware, <35B. However also bigger models are interesting for research purposes. Great image generation models released this year were Flux and Grok Aurora (not yet open-source).
An awesome paper from this year was “Coconut (Chain of Continuous Thought): Training Large Language Models to Reason in a Continuous Latent Space” from Meta, I believe this is a much better way to do “reasoning” (are we there yet?).
Yearly themes: “Year of Machine Learning”, “Year of Descipline”
I want to learn more mathematics, physics and computer science. I want to delve deeper into classical ML models, and after I feel comfortable with this foundation I want to move on to deep learning in much greater detail. AI (and its prerequesities) is going to be my main focus this year, I want to learn a lot. Hopefully, next year I will also have “favorite papers/talks” sections!
If I get time, I want to do a project involving GPU Programming / CUDA / Vulkan. I got an Nvidia Jetson (the old one) and I want to use it for something cool. Besides, low level GPU knowledge is useful besides graphics, I am interesting in stuff such as physics simulations and deep learning optimization methods, custom kernels, etc. A bit hard to balance since school is taking quite a lot of time with completly useless stuff, but I’ll manage it.
Will 2025 be the year of “AI Agents”? I think so but not because AGI is here, cause it’s not. I think LLMs have just gotten decent enough at pattern recognition so they are usable in more SaaSs. We might only see one, at most two, big products. I think at the moment Perplexity is the only big good product based on “agents”. I will also explore this space.
Happy New Year!