Programming with LLMs (Interview)
1Digest
This podcast features a conversation with David Krosha, a developer with extensive experience using LLMs for programming, and Scott Dietzen. They discuss the benefits and challenges of integrating LLMs into various programming workflows. Krosha shares his experiences at Tailscale and his work on Sketch.dev, a tool designed to improve LLM integration in Go. The conversation covers various aspects, including prompt engineering, the cognitive load associated with using LLMs, the comparison of different LLM types (chat-based vs. code completion), and the importance of context and clear instructions when interacting with AI models. They also touch upon the future of AI-assisted development, the role of tools like Augment Code, and the impact of LLMs on programming language design. Practical advice is given for programmers looking to incorporate AI tools into their workflows, recommending starting with code completion engines and gradually progressing to more complex applications. The discussion also highlights the importance of effective prompting techniques and the influence of user demeanor on AI interaction.
Outlines
Introduction and David Krosha's Early LLM Experiences
Introduction to the podcast and guest David Krosha, who shares his initial experiences using LLMs for programming and the benefits he's observed. A brief advertisement for fly.io is included.
LLMs at Tailscale and Multi-Cloud Challenges
David discusses his past role at Tailscale and why LLMs weren't a good fit for their product at that time, explaining the complexities of LLM network traffic and multi-cloud environments.
Augment Code and AI-Assisted Development
Introduction to Augment Code, an AI assistant for codebases, and a comparison with GitHub Copilot, highlighting Augment Code's superior context awareness.
David Krosha's LLM Journey and Cognitive Load
David details his year-long experience with LLMs, emphasizing the significant cognitive load involved and the need for better tooling. He discusses code completion, chat systems, and production workflow integration challenges.
Sketch.dev and Improved LLM Integration in Go
Discussion of Sketch.dev, a tool improving LLM integration in Go, focusing on prompt engineering and tool use to enhance LLM capabilities and reduce direct chat interactions.
Temporal, Programming Languages, and LLM Challenges
Discussion of Temporal, a platform for building resilient applications, and a comparison to the evolution of document editing. The conversation touches upon suitable programming languages for LLMs and integration challenges.
AI in Programming: Real-Time Ads and Code Completion
Discussion of Intel Xeon processors and the use of AI in programming, specifically in real-time ad pulling and code completion.
Getting Started with AI for Programming: Practical Advice
Practical advice for programmers starting with AI tools, recommending code completion engines and chat interfaces for smaller tasks, mentioning various AI models (ChatGPT, Claude, Llama CPP).
Effective Prompting Techniques for AI
Discussion of effective prompting strategies, suggesting treating AI like a new team member needing clear instructions and context, emphasizing iterative refinement.
User Demeanor and AI Interaction
Exploration of the impact of user attitude on AI interaction, discussing whether being "nice" improves results and its implications for human interaction.
Keywords
Large Language Models (LLMs)
Sophisticated AI models generating human-like text, used in code completion, chatbots, etc.
Code Completion
AI-powered feature suggesting code completions, improving efficiency and reducing errors.
Prompt Engineering
Crafting effective prompts to elicit desired responses from LLMs.
AI-Assisted Development
Using AI tools to augment software development processes.
Multi-cloud Environments
IT infrastructure using services from multiple cloud providers.
Augment Code
AI assistant deeply understanding codebases, superior to GitHub Copilot.
Sketch.dev
Tool improving LLM integration in Go programming.
ChatGPT
Popular large language model chatbot developed by OpenAI.
Llama CPP
Open-source large language model runnable locally.
Q&A
What are the main challenges in integrating LLMs into daily programming workflows?
Significant cognitive load, need for improved tooling, and rapid technological evolution.
How do tools like Sketch.dev aim to improve the LLM user experience for programmers?
By automating manual steps, reducing reliance on direct chat interactions, and focusing on integrated workflows.
What are the advantages and disadvantages of using different types of LLMs (e.g., chat-based vs. code completion)?
Chat-based models offer flexibility but can be slow; code completion models are faster but less flexible.
How can I improve my interactions with AI models to get better results?
Treat the AI like a new team member; provide clear context, background information, and iterative feedback.
Is it beneficial to be "nice" to AI models?
Kindness might not directly improve performance, but it can positively influence your problem-solving approach.
Show Notes
For the past year, David Crawshaw has intentionally sought ways to use LLMs while programming, in order to learn about them. He now regularly use LLMs while working and considers their benefits a net-positive on his productivity. David wrote down his experience, which we found both practical and insightful. Hopefully you will too!
Changelog++ members get a bonus 11 minutes at the end of this episode and zero ads. Join today!
Sponsors:
- Retool – The low-code platform for developers to build internal tools — Some of the best teams out there trust Retool…Brex, Coinbase, Plaid, Doordash, LegalGenius, Amazon, Allbirds, Peloton, and so many more – the developers at these teams trust Retool as the platform to build their internal tools. Try it free at retool.com/changelog
- Augment Code – Developer AI that uses deep understanding of your large codebase and how you build software to deliver personalized code suggestions and insights. Augment provides relevant, contextualized code right in your IDE or Slack. It transforms scattered knowledge into code or answers, eliminating time spent searching docs or interrupting teammates.
- Temporal – Build invincible applications. Manage failures, network outages, flaky endpoints, long-running processes and more, ensuring your workflows never fail. Register for Replay in London, March 3-5 to break free from the status quo.
Featuring:
- David Crawshaw – Website, GitHub, Bluesky, X
- Adam Stacoviak – Website, GitHub, LinkedIn, Mastodon, X
- Jerod Santo – Website, GitHub, LinkedIn, Mastodon, X
Show Notes:
- How I program with LLMs
- sketch.dev
- merde.ai: ai-fixed merge conflicts
- How Tailscale’s free plan stays free
Something missing or broken? PRs welcome!
Table of contents
United States
