The Pragmatic Engineer

Brought to you by:• Paragon: ​​Build native, customer-facing SaaS integrations 7x faster.• WorkOS: For B2B leaders building enterprise SaaS—On today’s episode of The Pragmatic Engineer, I’m joined by Quinn Slack, CEO and co-founder of Sourcegraph, a leading code search and intelligence platform. Quinn holds a degree in Computer Science from Stanford and is deeply passionate about coding: to the point that he still codes every day! He also serves on the board of Hack Club, a national nonprofit dedicated to bringing coding clubs to high schools nationwide. In this insightful conversation, we discuss:            • How Sourcegraph's operations have evolved since 2021• Why more software engineers should focus on delivering business value• Why Quinn continues to code every day, even as a CEO• Practical AI and LLM use cases and a phased approach to their adoption• The story behind Job Fairs at Sourcegraph and why it’s no longer in use• Quinn’s leadership style and his focus on customers and product excellence• The shift from location-independent pay to zone-based pay at Sourcegraph• And much more!—Where to find Quinn Slack:• X: https://x.com/sqs• LinkedIn: https://www.linkedin.com/in/quinnslack/• Website: https://slack.org/Where to find Gergely:• Newsletter: https://www.pragmaticengineer.com/• YouTube: https://www.youtube.com/c/mrgergelyorosz• LinkedIn: https://www.linkedin.com/in/gergelyorosz/• X: https://x.com/GergelyOrosz—In this episode, we cover:(01:35) How Sourcegraph started and how it has evolved over the past 11 years(04:14) How scale-ups have changed (08:27) Learnings from 2021 and how Sourcegraph’s operations have streamlined(15:22) Why Quinn is for gradual increases in automation and other thoughts on AI(18:10) The importance of changelogs(19:14) Keeping AI accountable and possible future use cases (22:29) Current limitations of AI(25:08) Why early adopters of AI coding tools have an advantage (27:38) Why AI is not yet capable of understanding existing codebases (31:53) Changes at Sourcegraph since the deep dive on The Pragmatic Engineer blog(40:14) The importance of transparency and understanding the different forms of compensation(40:22) Why Sourcegraph shifted to zone-based pay(47:15) The journey from engineer to CEO(53:28) A comparison of a typical week 11 years ago vs. now(59:20) Rapid fire roundThe Pragmatic Engineer deepdives relevant for this episode:• Inside Sourcegraph’s engineering culture: Part 1 https://newsletter.pragmaticengineer.com/p/inside-sourcegraphs-engineering-culture• Inside Sourcegraph’s engineering culture: Part 2 https://newsletter.pragmaticengineer.com/p/inside-sourcegraphs-engineering-culture-part-2—References and Transcript:See the transcript and other references from the episode at https://newsletter.pragmaticengineer.com/podcast—Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email podcast@pragmaticengineer.com. Get full access to The Pragmatic Engineer at newsletter.pragmaticengineer.com/subscribe

The first episode of The Pragmatic Engineer Podcast is out. Expect similar episodes every other Wednesday. You can add the podcast in your favorite podcast player, and have future episodes downloaded automatically.Listen now on Apple, Spotify, and YouTube.Brought to you by:• Codeium: ​​Join the 700K+ developers using the IT-approved AI-powered code assistant.• TLDR: Keep up with tech in 5 minutes—On the first episode of the Pragmatic Engineer Podcast, I am joined by Simon Willison.Simon is one of the best-known software engineers experimenting with LLMs to boost his own productivity: he’s been doing this for more than three years, blogging about it in the open.Simon is the creator of Datasette, an open-source tool for exploring and publishing data. He works full-time developing open-source tools for data journalism, centered on Datasette and SQLite. Previously, he was an engineering director at Eventbrite, joining through the acquisition of Lanyrd, a Y Combinator startup he co-founded in 2010. Simon is also a co-creator of the Django Web Framework. He has been blogging about web development since the early 2000s.In today’s conversation, we dive deep into the realm of Gen AI and talk about the following: • Simon’s initial experiments with LLMs and coding tools• Why fine-tuning is generally a waste of time—and when it’s not• RAG: an overview• Interacting with GPTs voice mode• Simon’s day-to-day LLM stack• Common misconceptions about LLMs and ethical gray areas • How Simon’s productivity has increased and his generally optimistic view on these tools• Tips, tricks, and hacks for interacting with GenAI tools• And more!I hope you enjoy this episode.—In this episode, we cover:(02:15) Welcome(05:28) Simon’s ‘scary’ experience with ChatGPT(10:58) Simon’s initial experiments with LLMs and coding tools(12:21) The languages that LLMs excel at(14:50) To start LLMs by understanding the theory, or by playing around?(16:35) Fine-tuning: what it is, and why it’s mostly a waste of time(18:03) Where fine-tuning works(18:31) RAG: an explanation(21:34) The expense of running testing on AI(23:15) Simon’s current AI stack (29:55) Common misconceptions about using LLM tools(30:09) Simon’s stack – continued (32:51) Learnings from running local models(33:56) The impact of Firebug and the introduction of open-source (39:42) How Simon’s productivity has increased using LLM tools(41:55) Why most people should limit themselves to 3-4 programming languages(45:18) Addressing ethical issues and resistance to using generative AI(49:11) Are LLMs are plateauing? Is AGI overhyped?(55:45) Coding vs. professional coding, looking ahead(57:27) The importance of systems thinking for software engineers (1:01:00) Simon’s advice for experienced engineers(1:06:29) Rapid-fire questions—Where to find Simon Willison:• X: https://x.com/simonw• LinkedIn: https://www.linkedin.com/in/simonwillison/• Website: https://simonwillison.net/• Mastodon: https://fedi.simonwillison.net/@simon—Referenced:• Simon’s LLM project: https://github.com/simonw/llm• Jeremy Howard’s Fast Ai: https://www.fast.ai/• jq programming language: https://en.wikipedia.org/wiki/Jq_(programming_language)• Datasette: https://datasette.io/• GPT Code Interpreter: https://platform.openai.com/docs/assistants/tools/code-interpreter• Open Ai Playground: https://platform.openai.com/playground/chat• Advent of Code: https://adventofcode.com/• Rust programming language: https://www.rust-lang.org/• Applied AI Software Engineering: RAG: https://newsletter.pragmaticengineer.com/p/rag• Claude: https://claude.ai/• Claude 3.5 sonnet: https://www.anthropic.com/news/claude-3-5-sonnet• ChatGPT can now see, hear, and speak: https://openai.com/index/chatgpt-can-now-see-hear-and-speak/• GitHub Copilot: https://github.com/features/copilot• What are Artifacts and how do I use them?: https://support.anthropic.com/en/articles/9487310-what-are-artifacts-and-how-do-i-use-them• Large Language Models on the command line: https://simonwillison.net/2024/Jun/17/cli-language-models/• Llama: https://www.llama.com/• MLC chat on the app store: https://apps.apple.com/us/app/mlc-chat/id6448482937• Firebug: https://en.wikipedia.org/wiki/Firebug_(software)#• NPM: https://www.npmjs.com/• Django: https://www.djangoproject.com/• Sourceforge: https://sourceforge.net/• CPAN: https://www.cpan.org/• OOP: https://en.wikipedia.org/wiki/Object-oriented_programming• Prolog: https://en.wikipedia.org/wiki/Prolog• SML: https://en.wikipedia.org/wiki/Standard_ML• Stabile Diffusion: https://stability.ai/• Chain of thought prompting: https://www.promptingguide.ai/techniques/cot• Cognition AI: https://www.cognition.ai/• In the Race to Artificial General Intelligence, Where’s the Finish Line?: https://www.scientificamerican.com/article/what-does-artificial-general-intelligence-actually-mean/• Black swan theory: https://en.wikipedia.org/wiki/Black_swan_theory• Copilot workspace: https://githubnext.com/projects/copilot-workspace• Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems: https://www.amazon.com/Designing-Data-Intensive-Applications-Reliable-Maintainable/dp/1449373321• Bluesky Global: https://www.blueskyglobal.org/• The Atrocity Archives (Laundry Files #1): https://www.amazon.com/Atrocity-Archives-Laundry-Files/dp/0441013651• Rivers of London: https://www.amazon.com/Rivers-London-Ben-Aaronovitch/dp/1625676158/• Vanilla JavaScript: http://vanilla-js.com/• jQuery: https://jquery.com/• Fly.io: https://fly.io/—Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email podcast@pragmaticengineer.com. Get full access to The Pragmatic Engineer at newsletter.pragmaticengineer.com/subscribe

Welcome to The Pragmatic Engineer Podcast, hosted by Gergely Orosz, the author of The Pragmatic Engineer newsletter. In each episode, we dive deep into the world of software engineering, offering practical insights on scaling teams, engineering leadership, and navigating the evolving tech landscape. With industry veterans and successful engineers as guests, this podcast is perfect for anyone looking to level up their engineering career with real-world advice.Subscribe to the podcast on YouTube, on Spotify, or Apple.You can also subscribe to the newsletter here. Get full access to The Pragmatic Engineer at newsletter.pragmaticengineer.com/subscribe