Content Recommendations- 2/19/2025 [Updates]

What you should know in AI, Software, Business, and Tech

Hey, it’s Devansh 👋👋

In issues of Updates, I will share interesting content I came across. While the focus will be on AI and Tech, the ideas might range from business, philosophy, ethics, and much more. The goal is to share interesting content with y’all so that you can get a peek behind the scenes into my research process.

I put a lot of effort into creating work that is informative, useful, and independent from undue influence. If you’d like to support my writing, please consider becoming a paid subscriber to this newsletter. Doing so helps me put more effort into writing/research, reach more people, and supports my crippling chocolate milk addiction. Help me democratize the most important ideas in AI Research and Engineering to over 100K readers weekly. You can use the following for an email template.

PS- We follow a “pay what you can” model, which allows you to support within your means, and support my mission of providing high-quality technical education to everyone for less than the price of a cup of coffee. Check out this post for more details and to find a plan that works for you.

---

Before we get into the AI Stuff, I saw “Capt America Brave New World” today. Didn’t really go in with many expectations since I haven’t really cared about the MCU (it’s not bad, but whatever I’ve seen hasn’t been great enough for me to want to follow)- but was let down with the script. Writers really let down the “Brave New World” legacy. Curious what y’all thought of the movie. Anything you really like recently?

Onwards-

A lot of people reach out to me for reading recommendations. I figured I’d start sharing whatever AI Papers/Publications, interesting books, videos, etc I came across each week. Some will be technical, others not really. I will add whatever content I found really informative (and I remembered throughout the week). These won’t always be the most recent publications- just the ones I’m paying attention to this week. Without further ado, here are interesting readings/viewings for 2/19/2025. If you missed last time’s readings, you can find it here.

Reminder- We started an AI Made Simple Subreddit. Come join us over here- https://www.reddit.com/r/AIMadeSimple/. If you’d like to stay on top of community events and updates, join the discord for our cult here: https://discord.com/invite/EgrVtXSjYf. Lastly, if you’d like to get involved in our many fun discussions, you should join the Substack Group Chat Over here.

Community Spotlight: Sebastian Raschka, PhD

Sebastian Raschka, PhD is one of the GOATs of AI Education. I frequently recommend posts from his excellent Ahead of AI Publication (his recent breakdown of Reasoning Models is elite). His books are also some of the best written in the space, and they’re one of my goto gifts anytime I meet anyone who’s interested in understanding AI beyond just the basics. Seb has contributed a lot to the open research community in cutting-edge AI, and I think you’re missing out on a lot of key insights if you’re not following him.

If you’re doing interesting work and would like to be featured in the spotlight section, just drop your introduction in the comments/by reaching out to me. There are no rules- you could talk about a paper you’ve written, an interesting project you’ve worked on, some personal challenge you’re working on, ask me to promote your company/product, or anything else you consider important. The goal is to get to know you better, and possibly connect you with interesting people in our chocolate milk cult. No costs/obligations are attached.

Previews

Curious about what articles I’m working on? Here are the previews for the next planned articles-

Tech Made Simple

Fallacies

AI Made Simple-

Deepseek Technical Breakdown

I provide various consulting and advisory services. If you‘d like to explore how we can work together, reach out to me through any of my socials over here or reply to this email.

Highly Recommended

These are pieces that I feel are particularly well done. If you don’t have much time, make sure you at least catch these works.

Microsoft’s Majorana 1 chip carves new path for quantum computing

In December 29th of last year, I published the “6 AI Trends that will Define 2025” piece where I covered the technologies that I believed would breakthrough in 2025. Quatum Computing was the first field we picked. I picked QC due to it’s strong synergies with HPC, AI, and Synthetic Data- all of which are powerful and growing fields currently.

Microsoft’s newest publication is a powerful proof of our prediction. Their Quantum Lab specifically mentioned all the above fields as key drivers to the success of their new quantum chip, that is slated to bring the quantum revolution decades closer. Very cool work, super excited to see the results (and check out the trends article for more trends that will pop off this year). I’m not going to say anything- but when I make claims on what trends are important, I do often have inside information from some very high-level sources that I base them on.

In the same way that the invention of semiconductors made today’s smartphones, computers and electronics possible, topoconductors and the new type of chip they enable offer a path to developing quantum systems that can scale to a million qubits and are capable of tackling the most complex industrial and societal problems, Microsoft said.

“We took a step back and said ‘OK, let’s invent the transistor for the quantum age. What properties does it need to have?’” said Chetan Nayak, Microsoft technical fellow. “And that’s really how we got here — it’s the particular combination, the quality and the important details in our new materials stack that have enabled a new kind of qubit and ultimately our entire architecture.”

This new architecture used to develop the Majorana 1 processor offers a clear path to fit a million qubits on a single chip that can fit in the palm of one’s hand, Microsoft said. This is a needed threshold for quantum computers to deliver transformative, real-world solutions — such as breaking down microplastics into harmless byproducts or inventing self-healing materials for construction, manufacturing or healthcare. All the world’s current computers operating together can’t do what a one-million-qubit quantum computer will be able to do.

Evolving Deeper LLM Thinking

Another proof that our trends piece identified the right technology. In it, I pointed to Evolutionary Algorithms as a powerful search method since they can search through diverse search spaces and find some good solutions outside local minima.

We explore an evolutionary search strategy for scaling inference time compute in Large Language Models. The proposed approach, Mind Evolution, uses a language model to generate, recombine and refine candidate responses. The proposed approach avoids the need to formalize the underlying inference problem whenever a solution evaluator is available. Controlling for inference cost, we find that Mind Evolution significantly outperforms other inference strategies such as Best-of-N and Sequential Revision in natural language planning tasks. In the TravelPlanner and Natural Plan benchmarks, Mind Evolution solves more than 98% of the problem instances using Gemini 1.5 Pro without the use of a formal solver.

The Artificial Investor - Issue 45: Does AI make us dumber?

A very timely article by the insightful Aristotelis Xenofontos

A few days ago, researchers at Carnegie Mellon and Microsoft published a study on the impact of GenAI on critical thinking among knowledge workers. The research found that while GenAI reduces cognitive effort, it can also decrease critical thinking. Higher confidence in GenAI correlated with less critical thinking, which occurred more in younger participants.

Is GenAI making us dumber? Or rather do AI’s benefits overweigh the costs? Is this any different from what happened with other technological innovations? What does this mean about the future of Tech?

Let’s dive in.

Harvard team built a CMOS chip to map 70,000 synaptic connections between 2,000 rat neurons

No idea what the deep implications are here, but this is very cool. Thank you Daanish Daanish for the recommendation. Truly a 5/7 thought leader.

Harvard researchers built a complementary metal-oxide semiconductor (CMOS) chip packed with 4,096 microhole electrode arrays, allowing them to record electrical activity across several neural cells. According to the Nature journal, this enabled the team to plot 2,000 rat neurons and map over 70,000 connections between them, with the chip able to measure the signal strength between each connection and characterize the signal type being sent across them.

This is a massive advancement in neuronal research, where scientists can accurately map every detail of neural connections within the brain. At the moment, electron microscopy can visualize these synaptic connections, but it cannot measure and record the signals going across them. Another technique — patch-clamp electrode — allows researchers to accurately record even the faintest of neural signals. However, this technique only measures across a handful of cells, limiting its effectiveness in studying large numbers of neurons.

The new CMOS chip allows researchers to study how a relatively large number of neurons interact, allowing them to understand how their activity results in complex mental processes, like thinking and learning. The researchers said that each microhole is like a patch-clamp electrode; so, by adding over 4,000 of these arrays inside a single chip, they were able to effectively monitor thousands of neurons.

AI Job Pulse: Companies Make Finding AI Jobs Really Difficult

Good note on the Job Market by Logan Thorneloe

I’ve been scanning job listings to better understand the job market for the past few months and want to more regularly share my findings with you. The good news: AI listings are on the rise and most of those listings are looking for software engineers. Those jobs also pay very well.

The bad news: Most of these job listings are terrible. They don’t properly explain the role, don’t have the right requirements, or are looking for a ridiculous number of years of experience. They frequently use overloaded or vague terminology (i.e. “Machine Learning Engineer” without an explanation of what the role actually does)1. All of this combines for an abysmal job hunting experience.

How to Build Wealth as a Woman — Even If You’re a Caregiver

This was a new idea that I didn’t know about. I’d be interested in learning more about this, how big of a problem it is, how it usually manifests, how you even measure something like this, and how it can be fixed. So many interesting threads to pick at here. Phenomenal work by Maryann .

Also- this article contains this haunting line- “It shows up in their bodies. The women that are everything to everyone, except themselves, develop breast cancer as a cry for help.” Makes you really step back and wonder.

What if I told you there’s an invisible tax that drains 30% of women’s wealth, forces many to retire into financial insecurity, and is so ingrained in our culture that we hardly question it?

The Hidden Tax on Women’s Wealth

It’s not the wage gap.

It’s not the pink tax.

It’s caregiving — an unpaid, undervalued, and unprotected form of labour that disproportionately falls on women.

Every day, millions of women step away from their careers, decline promotions, or work reduced hours to care for children, ageing parents, and family members in need.

They do this…

…often without financial safeguards

…often without questioning whether they should

Because it has been ingrained in them: this is what women do.

The result? Women retire with one-third less wealth than men.

A reader of my recent Substack note summed it up in four words: “Current status: spent and empty.”

[ISSUE #14] LinkedIn’s LLM-powered text-to-SQL solution using agents 🤖

A great breakdown by Niraj Pandkar . He does some pretty good AI Engineering breakdowns, so would recommend reading him. His work is much more condensed than my breakdowns, which might just be the thing a lot of you are looking for. Niraj took a big break from writing, but I’m glad to see him back and I’m excited to see what he puts out again.

This issue is about how LinkedIn created a text-to-SQL bot, which is now used company-wide. According to their survey, 95% of the users have rated the bot’s accuracy as “passes” and 40% rated it as “very good” or “excellent” which instills some confidence in deploying such a bot in your organization!

This article outlines -

1. the motivation for text-to-SQL bot

2. the challenges LinkedIn faced

3. the solutions they came up with to combat the above challenges

4. the agentic workflow they used to generate queries

5. how they focussed on UI/UX for better adoption of the tool

6. the benchmark they created for the evaluation of the tool

Harmonic Loss Trains Interpretable AI Models

This looks like a major breakthrough, but I feel like I’m missing something that I haven’t quite caught. Would love it if a bunch of you read it and tell me what you think so my life becomes easier. My legal AI startup IQIDIS popped off a bit, so my priority has been on servicing our paying users. Haven’t been able to do much of the researchy research recently, so would appreciate any insights you have to share.

In this paper, we introduce harmonic loss as an alternative to the standard cross-entropy loss for training neural networks and large language models (LLMs). Harmonic loss enables improved interpretability and faster convergence, owing to its scale invariance and finite convergence point by design, which can be interpreted as a class center. We first validate the performance of harmonic models across algorithmic, vision, and language datasets. Through extensive experiments, we demonstrate that models trained with harmonic loss outperform standard models by: (a) enhancing interpretability, (b) requiring less data for generalization, and © reducing grokking. Moreover, we compare a GPT-2 model trained with harmonic loss to the standard GPT-2, illustrating that the harmonic model develops more interpretable representations. Looking forward, we believe harmonic loss has the potential to become a valuable tool in domains with limited data availability or in high-stakes applications where interpretability and reliability are paramount, paving the way for more robust and efficient neural network models.

Why Crocodiles Are Thriving in the Shadow of A Nuclear Plant

Another reason why Nuclear energy is GOATed.

Florida’s native crocodiles have found an unexpected sanctuary in the cooling canals at Turkey Point Nuclear Power Plant. This fortunate happenstance, along with tireless conservation efforts, are helping these once endangered predators make a remarkable comeback.

I Investigated The Disappearance of England’s Greatest Wonderkid

Seeing this video, it was shocking how viciously the English press tried to tear down Alli. I’m no Joe Rogan fan, but looking at how unscrupulously the media can behave- whether it’s in England with Wazza being another example of a player that was tormented aggressively, the clown show that is Indian Political Reporting, or the biased reporting I’ve seen here in America- it’s not shocking to see why so many people are growing to distrust it. Wonder what we can do to fix it.

Accelerated enzyme engineering by machine-learning guided cell-free expression

Enzyme engineering is limited by the challenge of rapidly generating and using large datasets of sequence-function relationships for predictive design. To address this challenge, we develop a machine learning (ML)-guided platform that integrates cell-free DNA assembly, cell-free gene expression, and functional assays to rapidly map fitness landscapes across protein sequence space and optimize enzymes for multiple, distinct chemical reactions. We apply this platform to engineer amide synthetases by evaluating substrate preference for 1217 enzyme variants in 10,953 unique reactions. We use these data to build augmented ridge regression ML models for predicting amide synthetase variants capable of making 9 small molecule pharmaceuticals. Over these nine compounds, ML-predicted enzyme variants demonstrate 1.6- to 42-fold improved activity relative to the parent. Our ML-guided, cell-free framework promises to accelerate enzyme engineering by enabling iterative exploration of protein sequence space to build specialized biocatalysts in parallel.

OpenAI, Part 2

Eric Flaningam serving brilliance as usual. This is my favorite one that he's done yet, although that might be my bias since he listed getting dinner with me as a recommended resource for learning about AI. If any of you want to avail that service, give me a shout.

Well, that’s not as easy as it sounds. OpenAI might be as complex as they are important.

The goal of this article is very simple: to explain OpenAI. Its technology. Its business model. Its current state today.

This article has three sections:

1. A Primer on OpenAI’s Technology (Deep Learning, LLMs, Reasoning, Agents)

2. A Breakdown of OpenAI’s Business (Business Model, Revenue, Competition)

3. Market Statistics & Competitive Landscape

If my first OpenAI article was its history, this is its present, and the final article will be its future (or at least the questions I’m thinking about for its future)

Uncovering the Impact of Chain-of-Thought Reasoning for Direct Preference Optimization: Lessons from Text-to-SQL

Text to SQL is a very powerful use case for Language Models (I’m also biased to it since that was my deep forway into OG LLMs back in Jan 2022). It’s always interesting to see what things people get upto.

Direct Preference Optimization (DPO) has proven effective in complex reasoning tasks like math word problems and code generation. However, when applied to Text-to-SQL datasets, it often fails to improve performance and can even degrade it. Our investigation reveals the root cause: unlike math and code tasks, which naturally integrate Chain-of-Thought (CoT) reasoning with DPO, Text-to-SQL datasets typically include only final answers (gold SQL queries) without detailed CoT solutions. By augmenting Text-to-SQL datasets with synthetic CoT solutions, we achieve, for the first time, consistent and significant performance improvements using DPO. Our analysis shows that CoT reasoning is crucial for unlocking DPO’s potential, as it mitigates reward hacking, strengthens discriminative capabilities, and improves scalability. These findings offer valuable insights for building more robust Text-to-SQL models. To support further research, we publicly release the code and CoT-enhanced datasets.

Understanding and Attacking Meta’s Chameleon Model

Max Buckley is based for contributing towards MultiModal Adversarial Perturbations. We need more Max’s in the world.

For this project, I decided to investigate adversarial attacks for Meta’s recent Chameleon model using the two modalities it presents — text and images. I chose this particular project for a variety of reasons: I wanted to learn more about vision language models (VLMs), and adversarial attacks aligns with some upcoming work I have planned, so any knowledge gleaned here will form a solid basis for my future projects at ETH.

Other Content

LB-KBQA:Large-language-model and BERT based Knowledge-Based Question and Answering System

Generative Artificial Intelligence (AI), because of its emergent abilities, has empowered various fields, one typical of which is large language models (LLMs). One of the typical application fields of Generative AI is large language models (LLMs), and the natural language understanding capability of LLM is dramatically improved when compared with conventional AI-based methods. The natural language understanding capability has always been a barrier to the intent recognition performance of the Knowledge-Based-Question-and-Answer (KBQA) system, which arises from linguistic diversity and the newly appeared intent. Conventional AI-based methods for intent recognition can be divided into semantic parsing-based and model-based approaches. However, both of the methods suffer from limited resources in intent recognition. To address this issue, we propose a novel KBQA system based on a Large Language Model(LLM) and BERT (LB-KBQA). With the help of generative AI, our proposed method could detect newly appeared intent and acquire new knowledge. In experiments on financial domain question answering, our model has demonstrated superior effectiveness.

Google DeepMind Introduces MONA: A Novel Machine Learning Framework to Mitigate Multi-Step Reward Hacking in Reinforcement Learning

Most existing methods use patching reward functions after detecting undesirable behaviors to combat these challenges. These methods are effective for single-step tasks but falter when avoiding sophisticated multi-step strategies, especially when human evaluators cannot fully understand the agent’s reasoning. Without scalable solutions, advanced RL systems risk producing agents whose behavior is unaligned with human oversight, potentially leading to unintended consequences.

Google DeepMind researchers have developed an innovative approach called Myopic Optimization with Non-myopic Approval (MONA) to mitigate multi-step reward hacking. This method consists of short-term optimization and long-term impacts approved through human guidance. In this methodology, agents always ensure that these behaviors are based on human expectations but avoid strategy that exploits far-off rewards. In contrast with traditional reinforcement learning methods that take care of an optimal entire task trajectory, MONA optimizes immediate rewards in real-time while infusing far-sight evaluations from overseers.

The core methodology of MONA relies on two main principles. The first is myopic optimization, meaning that the agents optimize their rewards for immediate actions rather than planning multi-step trajectories. This way, there is no incentive for the agents to develop strategies that humans cannot understand. The second principle is non-myopic approval, in which the human overseers provide evaluations based on the long-term utility of the agent’s actions as anticipated. These evaluations are, therefore, the driving forces for encouraging agents to behave in manners aligned with objectives set by humans but without getting direct feedback from outcomes.

Examining the Expanding Role of Synthetic Data Throughout the AI Development Pipeline

Alongside the growth of generative AI, we are witnessing a surge in the use of synthetic data across all stages of the AI development pipeline. It is now common practice for researchers and practitioners to use one large generative model (which we refer to as an auxiliary model) to generate synthetic data that is used to train or evaluate another, reconfiguring AI workflows and reshaping the very nature of data. While scholars have raised concerns over the risks of synthetic data, policy guidance and best practices for its responsible use have not kept up with these rapidly evolving industry trends, in part because we lack a clear picture of current practices and challenges. Our work aims to address this gap. Through 29 interviews with AI practitioners and responsible AI experts, we examine the expanding role of synthetic data in AI development. Our findings reveal how auxiliary models are now widely used across the AI development pipeline. Practitioners describe synthetic data as crucial for addressing data scarcity and providing a competitive edge, noting that evaluation of generative AI systems at scale would be infeasible without auxiliary models. However, they face challenges controlling the outputs of auxiliary models, generating data that accurately depict underrepresented groups, and scaling data validation practices that are based primarily on manual inspection. We detail general limitations of and ethical considerations for synthetic data and conclude with a proposal of concrete steps towards the development of best practices for its responsible use.

Controlled automatic task-specific synthetic data generation for hallucination detection

We present a novel approach to automatically generate task-specific synthetic datasets for hallucination detection. Our approach features a two-step generation-selection pipeline, where the generation step integrates a hallucination pattern guidance module and a language style alignment module. Hallucination pattern guidance makes it possible to curate synthetic datasets covering the most important hallucination patterns specific to target applications. Language style alignment improves the dataset quality by aligning the style of the synthetic dataset with benchmark text. To obtain robust supervised detectors from synthetic datasets, we also propose a data mixture strategy to improve performance robustness and model generalization. Our supervised hallucination detectors trained on synthetic datasets outperform in-context-learning (ICL)-based detectors by a large margin. Our extensive experiments confirm the benefits of our two-staged generation pipeline with cross-task and cross-hallucination pattern generalization. Our data-mixture-based training further improves generalization and the robustness of hallucination detection.

If you liked this article and wish to share it, please refer to the following guidelines.

Reach out to me

Use the links below to check out my other content, learn more about tutoring, reach out to me about projects, or just to say hi.

Small Snippets about Tech, AI and Machine Learning over here

AI Newsletter-

My grandma’s favorite Tech Newsletter-

My (imaginary) sister’s favorite MLOps Podcast-

Check out my other articles on Medium. : https://rb.gy/zn1aiu

My YouTube: https://rb.gy/88iwdd

Reach out to me on LinkedIn. Let’s connect: https://rb.gy/m5ok2y

My Instagram: https://rb.gy/gmvuy9

My Twitter: https://twitter.com/Machine01776819