Interesting Content in AI, Software, Business, and Tech- 11/8/2023 [Updates]
Content to help you keep up with Machine Learning, Deep Learning, Data Science, Software Engineering, Finance, Business, and more
Hey, it’s Devansh 👋👋
In issues of Updates, I will share interesting content I came across. While the focus will be on AI and Tech, the ideas might range from business, philosophy, ethics, and much more. The goal is to share interesting content with y’all so that you can get a peek behind the scenes into my research process.
If you’d like to support my writing, consider becoming a premium subscriber to my sister publication Tech Made Simple to support my crippling chocolate milk addiction. Use the button below for a discount.
p.s. you can learn more about the paid plan here. If you’re looking to work with me, feel free to reach out.
A lot of people reach out to me for reading recommendations. I figured I’d start sharing whatever AI Papers/Publications, interesting books, videos, etc I came across each week. Some will be technical, others not really. I will add whatever content I found really informative (and I remembered throughout the week). These won’t always be the most recent publications- just the ones I’m paying attention to this week. Without further ado, here are interesting readings/viewings for 11/8/2023. If you missed last week’s readings, you can find it here.
Reminder- We started an AI Made Simple Subreddit. Come join us over here- https://www.reddit.com/r/AIMadeSimple/
Community Spotlight- Nidhi Anarkat
Nidhi Anarkat does really cool with Navgurukul, a non-profit that helps girls from underprivileged backgrounds learn to code and helps them break into software jobs. These jobs can be life-changing, helping uplift families out of poverty. This helps level the playing field and bring more people into the fold (keep in mind these girls are excluded from universities and other educational opportunities). Nidhi will be coming to this newsletter to do a dedicated guest post about Navgurukul and their work- but in the meantime, if any of you are interested in speaking to her about possibly getting involved, please shoot her a message.
If you're doing interesting work and would like to be featured in the spotlight section, just drop your introduction in the comments/by reaching out to me. There are no rules- you could talk about a paper you've written, an interesting project you've worked on, some personal challenge you're working on, ask me to promote your company/product, or anything else you consider important. The goal is to get to know you better, and possibly connect you with interesting people in our chocolate milk cult. No costs/obligations are attached.
Highly Recommended
These are pieces that I feel are particularly well done. If you don't have much time, make sure you at least catch these works.
Generative AI is Breaking Society
Michael Spencer woke up and chose blood with this one. A great piece on the general disconnect, hysteria, and conflicting interests at the heart of the Gen AI.
"China's SMIC and Huawei have surged ahead in chip technology, defying the U.S. restrictions aimed at curbing their technological advancements, meanwhile the Biden Administration “Executive Order” on A.I. seems to favor the startups Microsoft, Amazon and Google itself have invested in. ...
People at OpenAI, like Ilya Sutskever said that humans could merge with AI (pictured above right) in the future, a transhumanistic reality most world citizens and consumers don’t actually want. So you have elites who call themselves scientists, without even understanding their creations, saying that the way forward for humanity is to merge with machines and machine intelligence. The near term future of alignment implications in LLMs is still uncertain. Generative A.I. has evoked the pyramid schemes Silicon Valley has used to perpetuate its AI supremacy and suposed appeal to consumers and enterprise customers. The caveat, there is no alternative to them and their services, whether that be in the Cloud, in digital Ads or in video platforms, channels, media and apps."
As a giant history nerd, this video caught me by surprise. Turns out that pretty much all of the popular narrative around Cleopatra is influenced by Romans who had a vested interest in painting Cleopatra to be this conniving seducer. In reality, Cleopatra was extremely intelligent, cared for her people, and was exceptionally good at conversations/negotiations. All facets of her that are conveniently overlooked b/c they would hurt the Roman Narrative. The reason most depictions focus on her sexuality is that Romans had to save face after two of their most prominent figures (Julius Ceaser and Mark Antony) fell madly in love with her. She was more than her romantic relationships, boobs, and pretty eyeliner. Her story goes to show how important it is to study the biases/influences in our data and systems.
The problem with Elon Musk’s ‘first principles thinking’
Charley Johnson does a great job covering why Objective Knowledge is a myth and that it's impossible for information itself to be truly neutral. Just as with the Cleopatra source, hopefully this makes you slower to make decisions and hold your ideas less strongly. All information transmitted prioritizes and discards certain contexts, which is why it's helpful to constantly expose ourselves to new ideas.
"Why? Well, absolute objectivity and value-neutral knowledge have always been a myth. Sergio Sismondo, a scholar of science and technology, explains absolute objectivity as, “the ideal of perfect knowledge of some object, knowledge that is true regardless of perspective.” This notion that we can employ a detached perspective to derive independent ideas is very similar to Thomas Nagel’s conception of ‘the view from nowhere’. This presumes that the truth is out there and that it’s the job of scientists and technologists to stand apart from the world to find it. ‘First principles thinking’ and ‘objectivity’ attempt to separate knowledge from where it comes from. But that’s where the problem lies: As Professor Donna Haraway once wrote, “All knowledge is knowledge from somewhere.” "
When Pharma met Artificial Intelligence
Marina T Alamanou, PhD writes great pieces covering AI and it's role in Pharma/Biotech. If that's a field you're interested in, highly recommend signing up to her newsletter.
"In the preclinical box candidate drugs are tested on small animals in cages in some basement and in an aseptic environment, while we all know that humans have a normal life in a normal environment the so-called “exposome,” (air particles, pollutants, viruses, and everything we come into contact with each day). While in the clinical box candidate drugs are tested on humans and well-designed experiments are carried out in laboratories.
Imagine now each of these clinical boxes full of a finite number of tiny smaller boxes. Each smaller box represents a patient or a lab experiment (more or less). For each of these smaller boxes, pharmas have built around them huge protective shields 🛡️ (regulations) in order to monitor every parameter that can affect — and they don’t want that — the millions (even billions) of variables inside each smaller box. But when it comes to the discovery and the preclinical box, the finite number of these tiny smaller boxes (that is, variables of the system) are left almost without regulations, apart some good laboratories practises that scientists should follow."
AI Engineering 101 Explained: Devansh
I was featured on the AI Healthcare Podcast. We spoke about AI risks, regulations that would be important for Machine Learning in Healthcare, and more. It was a lot of fun- and there are several important ideas we touch upon there. PS: There was a slight issue with recording, and the audio might be a bit choppy. But the ideas discussed are still super important and worth listening to
Why Lululemon Is Ahead of Its Time
A while back, we covered Under Armor and how they failed because they failed to really understand their market and business. Instead, they spent 700 Million Dollars, chasing trends. Lululemon, another athleisure brand, is a great example of the process done right.
“Elon’s Gonna Elon,” xAI Crashes Altman’s Party
The Tech Buzz is a great resource to those of you just looking to keep up with business-relevant news in the Tech Industry. Check them out if you're drowning in the updates.
How Evolution Hacked its way to Intelligence from the Bottom Up
A fantastic piece on how Modularity (the ability for cells to adapt to circumstances) might have the key to intelligence. If this is true, then seeking 'AI emergence' by scaling up seems to be the opposite to what we should be doing. Shoutout to LaSalle Browne for this great share.
"This is intelligence in action: the ability to reach a particular goal or solve a problem by undertaking new steps in the face of changing circumstances. It’s evident not just in intelligent people and mammals and birds and cephalopods, but also cells and tissues, individual neurons and networks of neurons, viruses, ribosomes and RNA fragments, down to motor proteins and molecular networks. Across all these scales, living things solve problems and achieve goals by flexibly navigating different spaces – metabolic, physiological, genetic, cognitive, behavioural.
But how did intelligence emerge in biology? The question has preoccupied scientists since Charles Darwin, but it remains unanswered. The processes of intelligence are so intricate, so multilayered and baroque, no wonder some people might be tempted by stories about a top-down Creator. But we know evolution must have been able to come up with intelligence on its own, from the bottom up."
I was featured on Andrew Smiths newsletter, where we talked about the Turk- the first prominent AI Scam that I know of. Went over how many of the techniques in that scam were very similar to today's hype marketing.
"This was the Turk, an elaborate automation that could win at chess against some of the best players in the world. Imagine how astounding this feat must have seemed two centuries ago, before the industrial revolution had taken hold in Europe, as Napoleon’s soldiers’ buttons were beginning to rot.
In an age where machinery was often rudimentary and predictable, the Turk was an enigma, a spectacle that defied understanding. It represented the promise of a future where technological wonders could surpass human capabilities. Those lucky enough to see it were witnessing the future.
Too bad it was all fake."
Other Content
Whether LLMs can "understand" and introducing a smarter way to chunk PDFs
"You could naively chunk your documents - a straightforward method of breaking down large documents into smaller text chunks without considering the document's inherent structure or layout.
Going this route, you end up dividing the text based on a predetermined size or word count, such as fitting within the LLM context window (typically 2000-3000 words). The problem is that you can disrupt the semantics and context implied by the document's structure.
Ambika Sukla proposes a solution called "smart chunking" that is layout-aware and considers the document's structure.
This method:
Is aware of the document's layout structure, preserving the semantics and context.
Identifies and retains sections, subsections, and their nesting structures.
Merges lines into coherent paragraphs and maintains connections between sections and paragraphs.
Preserves table layouts, headers, subheaders, and list structures.
To this end, he’s created the LlamaSherpa library, which has a "LayoutPDFReader," a tool designed to split text in PDFs into these layout-aware chunks, providing a more context-rich input for LLMs and enhancing their performance on large documents."
Rotary Positional Embeddings: Combining Absolute and Relative
In this video, I explain RoPE - Rotary Positional Embeddings. Proposed in 2022, this innovation is swiftly making its way into prominent language models like Google's PaLM and Meta's LLaMa. I unpack the magic behind rotary embeddings and reveal how they combine the strengths of both absolute and relative positional encodings.
A Linear Algebra Trick for Computing Fibonacci Numbers Fast
"I’ve been attending an online reading group on the book “Thirty-three Miniatures: Mathematical and Algorithmic Applications of Linear Algebra” lead by Prof. Neeldhara Misra from IIT Gandhinagar. It is the most unconventional math book I’ve come across. The first two chapters are all about ways to quickly compute Fibonacci numbers. The conventional (memoization based, or iterative) method of computing Fibonacci numbers that we learn in our programming courses is linear in time. But, the book shows a technique for computing them in approximately logarithmic time complexity. It’s possible that some of you might be aware of this technique but this was news to me and I thought it’s worth sharing with the rest of you."
Understanding LLMs Through The Problem Theyre Trained To Solve (paper breakdown)
The recursion formula behind life itself?
New research explores why college students overuse short-video platforms
"The study further explored how needs like knowledge acquisition, social identity, and entertainment drive behaviors such as social interaction and compensatory expectation. Additionally, the researchers examined the role of inhibitory control – the capacity for self-discipline – in moderating the relationship between these variables and app overuse.
In the study, 197 college students recruited from China and United States participated by completing online self-administered surveys. The findings revealed that while the entertainment needs of Chinese students did not notably impact their social interaction on short-video apps, American students’ need for entertainment negatively affected their social interaction on these platforms."
If you liked this article and wish to share it, please refer to the following guidelines.
If you find AI Made Simple useful and would like to support my writing- please consider becoming a premium member of my cult by subscribing below. Subscribing gives you access to a lot more content and enables me to continue writing. This will cost you 400 INR (5 USD) monthly or 4000 INR (50 USD) per year and comes with a 60-day, complete refund policy. Understand the newest developments and develop your understanding of the most important ideas, all for the price of a cup of coffee.
Reach out to me
Use the links below to check out my other content, learn more about tutoring, reach out to me about projects, or just to say hi.
Small Snippets about Tech, AI and Machine Learning over here
AI Newsletter- https://artificialintelligencemadesimple.substack.com/
My grandma’s favorite Tech Newsletter- https://codinginterviewsmadesimple.substack.com/
Check out my other articles on Medium. : https://rb.gy/zn1aiu
My YouTube: https://rb.gy/88iwdd
Reach out to me on LinkedIn. Let’s connect: https://rb.gy/m5ok2y
My Instagram: https://rb.gy/gmvuy9
My Twitter: https://twitter.com/Machine01776819
I love that you called out the Turk as the "first prominent AI scam."