1 year special: Ask Me Anything
Happy 1st birthday to our cult.
Exactly one year ago today I started this newsletter. It has been an absolutely crazy year, and I am always surprised, overwhelmed, and a little unnerved by the amount of support I have received. I never expected things to go as well as they did, and I feel lucky to have the opportunity to interact with AI/Tech leaders from all over the world.
I’ve seen many creators host AMAs to celebrate major milestones and thought this might be fun to try. I also receive quite a few questions from y’all, so this would be a good place to answer the ones I get regularly. We got a lot of submissions for this AMA, so let me know what you think of this format and if we should do this more often.
The questions we will answer today are-
What Tools do you use for productivity? How do you keep up with all the developments?
I am a manager with 20+ years of work experience in software and finance. Recently my team has been exploring AI. What courses do you recommend? Is getting a master’s degree worth it? (we will also touch upon this question from the perspective of early career folk).
What social medium should I focus on to publicize my newsletter? What is the “social media ladder”, if there is one at all, that I can climb on and how can I do that efficiently? I write here - nirajpandkar.Substack.com
When do you think we might see the dawn of sentient AI?
I love your logos. Is there a story behind them?
What made you start writing? Do you think it is beneficial for students to spend time on these projects?
How do you find time to research and do your job?
I'm a new student. What AI fields should I focus on for best career?
Who are your favorite writers?
What are some overlooked techniques you would recommend ML Engineers to learn?
Answering your questions
What Tools do you use for productivity? How do you keep up with all the developments?
One thing that surprises a lot of people is how low-tech my work setup is. I rely extensively on social media feeds and my network to find ideas for my next articles. I follow a lot of high-quality sources in AI, so reading their work also allows me to stay updated. Combined with this, I search for sources to study that I am interested in learning (such as Complex Valued NNs). This is how I get all my information to base my articles on. As you can see, the only AI I use to track the developments in AI are the social media recommendation algorithms and search engines.
When it comes to writing, I don’t really use any AI Tools (they have been pretty bad IMO). I use LLMs from time to time, but they don’t work for most of my articles.
The number 1 productivity tool I have is the Video Speed Controller app on Chrome. Aside from music and sports, I watch everything on 5x speed (anime, podcasts, analysis videos, movies, and anything else). If something has subtitles, I can go up to 6x, depending on what I’m watching (my fastest is One Piece at 6.5-7x speed). This saves me a lot of time (it takes me 4 minutes to watch a 20-minute video). Aside from these, I don’t really use any tools or subscriptions.
The important thing with a field like AI is to give up on being perfect. You will always miss something, not know about some important component etc. Instead of trying to keep up with ‘every development’ take things easy. Study what you can, track your industry very well, and join a bunch of discussion groups/resources to come across new ideas. That’s the best you can realistically do.
I am a manager with 20+ years of work experience in software and finance. Recently my team has been exploring AI. What courses do you recommend? Is getting a master’s degree worth it?
Variants of this question are some of the most common questions I get asked. When you have a work experience in your field, then a degree is not really going to help you. Your time is limited, so you’re better off monitoring trends, understanding your field (finance in this case), and understanding how the various components of the system work together. A degree will give you lot of insight into the inner workings of the AI, but that is not your money-maker. It is more important for you to have a big-picture understanding of the entire business process and trust your engineers to have the details figured out.
That being said, some theoretical understanding does help. My recommendation is to teach yourself. There are lots of free resources online that you can check out. I’ve created a list of some of my favorites here. My weekly content and Substack recommendations are also useful over there. For a manager (especially an experienced one), my recommendation is to focus on the overarching principles as opposed to the details (understand the basic idea behind gradient descent but don’t bother learning to implement it from scratch). If you are looking for someone to help you identify what to focus on, I’m always happy to talk.
On a related note, a lot of early career/university people ask me similar questions. For them, the question is a bit more nuanced. You can break into AI w/o an advanced degree (I did it). BUT it’s a very painful journey. I also got very lucky that my writing worked out. If it hadn’t, I would have applied for a PhD. The orignal purpose of my writing was to be a showcase of my knowledge, so that I could skip a Master’s Degree and go straight to PhD. The ROI of a degree, especially when targeting research-related roles (or roles at big companies) is huge.
Alternatively, you can pursue alternative roles (data engineering, cloud engineering, software etc) and try to transition into ML Engineering from there (switch internally, get a few years of work experience on your resume and you’re good). This is the path I will recommend if you don’t like school (it’s similar to what I did). This will get you working much sooner, but will restrict you from entering academia/research, so keep that in mind.
I did a video on this topic down below. Check if it interests you.
What social medium should I focus on to publicize my newsletter? What is the “social media ladder”, if there is one at all, that I can climb on and how can I do that efficiently? I write here - nirajpandkar.Substack.com
Every social media platform will match a different style. The way I see it, there are two kinds of social media platforms-
Discovery-based platforms: YouTube, TikTok, Medium. These platforms are great for reaching lots of people, especially when you don’t have an audience. However, the flip side to this is that these platforms aren’t the best for nurturing an audience. There is no guarantee that people will see your work, even if they follow you. Leverage these platforms to get readers onto the second kind of platform.
Retention-based platforms: Substack and IG are extreme examples of this. With such platforms, most of your content will go to your preexisting followers (like this email newsletter). This is great for building your presence with your readers, but also means that growth will be much slower.
LinkedIn for me lies in the middle, where it does both (more of the second than the first though). If you are interested in learning about my audience-building insights for LinkedIn, I did a breakdown of my journey to LinkedIn's top voice status here.
I have no idea about X (I only have 980 followers on X, so I can’t claim any kind of insight there). Despite my best efforts, I can’t really seem to crack engagement there. There is a lot of value in engaging with bigger creators there (more so than other platforms, except for maybe Substack), but have nothing more to say about it. To those of you who have found growth on X, what has your experience with the platform been? Would love to hear it.
When do you think we might see the dawn of sentient AI?
I would hope within the next 3-4 years. There are some interesting works with use actual brains in systems instead of ANNs. The combination of biological and artificial systems would be super cool to see. If you can call that sentience, then hope to see it soon. I’m surprised that there were no major venture capital investments into this now.
I love your logos. Is there a story behind them?
Thank you. My pfp is an AI-stylized version of an actual picture of me when I went camping in Tenessee (to catch a meteor shower). The dog in the picture is my friend’s.
The background has nothing particular. It represented the constant journey of learning to develop mastery in Tech and AI. Also, I love cats and someone suggested I include a kitty in my background.
What made you start writing? Do you think it is beneficial for students to spend time on these projects?
As I mentioned, I wanted to get a PhD. The writing was done as a showcase of my knowledge. The plan was to share the relevant articles I did when reaching out to professors (which is why I bounced between various kinds of topics). And some professors really appreciated my writing (although some didn’t like it, since they found my style of writing and memes unprofessional). Both the longer internships that I did in university also hired me because of my writing, when I didn’t have an audience. Content creation is worth experimenting with (even if it doesn’t work out, I think it can teach you a lot about yourself and your strengths).
If content creation doesn’t work, you can also use the skills you develop in your work. The work you do for content creation will also expose you to various communities, which you can use to supercharge your professional networks and find interesting open-source projects. There is a lot of secondary benefits to be gained from this journey, and the downside is pretty limited.
How do you find time to research and do your job?
I work as an AI consultant (both strategy and implementation) for multiple clients. My work involves a lot of reading AI Research/market news, which makes it much easier to write. It takes me around 3 hours to write most pieces (lesser to do the content lists) so I can find time to do it throughout the week. This takes a lot less effort than people realize since so much of the required research/networking is part of my job as an AI Engineer and Strategy Consultant. I haven’t had to work crazy hard to make this happen (just spent some time consistently over the last 3.5 years on it).
I'm a new student. What AI fields should I focus on for best career?
You should study what interests you the most. This might seem like a trivial strategy but this is effective for many reasons. Firstly, this way you prevent burnout better. If you’re looking only into ideas and topics you’re interested in, you will not spend a lot of time forcing yourself to grind through things you don’t care about. Mentally, this will free a lot of energy. Self-guided Research isn’t something you will be able to do full-time as you’re starting out. And you will have to invest a lot of time and effort into it before you start seeing major results. If you burn yourself out before that point, you will have just put in a lot of effort for no return.
This has another benefit- you will be able to dig much deeper into topics. If you’re genuinely interested in an idea, you will be able to put a lot more effort into researching the intricacies. You don’t have to care about everything, but focusing on studying to solve a problem that interests you is a good strategy to keep yourself engaged long-term.
Who are your favorite writers?
All time- Kierkegaard, Camus, and Kafka would be my top 3. Their writings have a pretty special place in the way I see the world. Some other people who influenced me quite a bit are: Peter Singer (“Famine, Affluence, and Morality” is something that I think about a lot); Machiavelli; and Doestevesky (especially The Idiot and Brothers Karamazov).
Other than that, Don Quixote will always be on the top of my reading list. If we expand writers to include mangakas then Berserk, Vagabond, and Gintama are in my top 5 works of art.
What are some overlooked techniques you would recommend ML Engineers to learn?
Here are 3 techniques that I believe any ML Engineer should know.
Randomization
To anyone who has been following my work for a while, this will not surprise you. In terms of benefits, implementing a degree of randomness into your Machine Learning Pipelines will improve your overall network- in terms of performance, robustness, and even costs. For eg, Google Researchers were able to beat SOTA image-classification models while using 12 times fewer images (and not needing to label these images).
We show a counter-intuitive result that adding more sources of variation to an imperfect estimator approaches better the ideal estimator at a reduction in compute 51xcost.
From the paper, “Accounting for Variance in Machine Learning Benchmarks”. Read my breakdown of that paper here
Why does this happen? The most likely explanation is that adding an element of randomness and chaos into your training protocols actually expands the solution space your model searches through. Thus the final model configurations will be more suited to generalize to a greater possible set of inputs. This helps a lot when it comes to building solutions that can generalize to the often chaotic and messy input you will see in the real world.
Evolutionary Methods
Speaking of things that can traverse a very diverse search space- it’s time to cover possibly the most overlooked tool in a Machine Learning Engineers arsenal. The amount of disrespect on EAs is unbelievable. The number of ML people I’ve talked to, who’ve never even considered evolutionary methods is staggering. It’s time to change that.
Evolutionary Algorithms are amazing for many reasons- they are cheap, can work on a more diverse set of problems than neural networks, they have shown amazing performance, and their training can easily be followed and analyzed.
Bayesian Learning
Most people in Machine Learning don’t understand Math and thus aren’t fully comfortable with (or even aware of) Bayesian Statistics and Causal Inference. A lot of the ‘cool’ ML research doesn’t cover this topic a lot, but Bayesian Learning has been a game changer to my effectiveness in ML.
Bayesian Learning can be a tricky thing to learn and understand, but it will be a great addition to your arsenal. I only really got good with it through my work on Supply Chain Analysis at ForeOptics(shoutout to my manager Shaheen for all his help with this), but since then, I’ve been able to solve some pretty interesting problems. To develop your skills with it, search up some talks/papers using Bayesian Learning to understand some contexts it’s being applied in (and why it works there). Use that to gain an appreciation for the subject. Then (and only then) look into the Math and coding of the topic.
We will end this AMA over here. Let me know what you think and if we should continue to
Reach out to me
Use the links below to check out my other content, learn more about tutoring, reach out to me about projects, or just to say hi.
Small Snippets about Tech, AI and Machine Learning over here
AI Newsletter- https://artificialintelligencemadesimple.substack.com/
My grandma’s favorite Tech Newsletter- https://codinginterviewsmadesimple.substack.com/
Check out my other articles on Medium. : https://rb.gy/zn1aiu
My YouTube: https://rb.gy/88iwdd
Reach out to me on LinkedIn. Let’s connect: https://rb.gy/m5ok2y
My Instagram: https://rb.gy/gmvuy9
My Twitter: https://twitter.com/Machine01776819
Congrats on the year completed! Great work.
Do you really watch videos at 5x speed?
How did you work up to that speed?
By 2.5 x I’m hearing chipmonks!
Congrats on a year!