Miki Lombardi

Automate everything via NodeJS and Puppeteer

How do you automate tasks on sites without a public API? See how NodeJS and Puppeteer can scrape data, publish content, and run continuous performance tests.

Automate everything via NodeJS and Puppeteer
#1about 2 minutes

An introduction to the Puppeteer library for Node.js

Puppeteer provides a high-level API to control headless or full Chrome instances through the DevTools Protocol.

#2about 2 minutes

The history and evolution of web automation tools

Web automation has evolved from cross-language frameworks like Selenium to JavaScript-based tools like PhantomJS and now Puppeteer.

#3about 3 minutes

Understanding Puppeteer's API and common capabilities

Puppeteer's readable, action-based API can be used for scraping, testing, generating PDFs, and analyzing performance.

#4about 3 minutes

Comparing Puppeteer with alternatives like Playwright

Playwright is a strong alternative for cross-browser end-to-end testing, while Puppeteer is more of a general-purpose automation tool.

#5about 4 minutes

Exploring common Puppeteer use cases for web applications

Puppeteer is well-suited for scraping dynamic sites, automating UI and visual regression tests, and building custom development utilities.

#6about 7 minutes

Case study on scraping Skyscanner for cheap flights

A personal project demonstrates how to automate flight searches by splitting code into modular functions and intercepting API responses.

#7about 5 minutes

Automating podcast uploads from YouTube to Anchor.fm

A GitHub Action uses Puppeteer to automate converting YouTube videos to audio and publishing them on Anchor.fm, saving significant manual effort.

#8about 7 minutes

Automating Lighthouse performance tests with authentication

A professional setup uses Puppeteer to handle authentication and run Lighthouse tests on protected pages, sending metrics to Prometheus and Grafana.

#9about 9 minutes

Q&A on Puppeteer vs Cypress and cloud deployment

The discussion covers when to choose Puppeteer over Cypress, running it in the cloud using containers instead of serverless, and its use in visual testing.

Related jobs
Jobs that call for the skills explored in this talk.

Matching moments

Automating browser workflows with AI-powered tools
03:32 MIN

Automating browser workflows with AI-powered tools

WeAreDevelopers LIVE: Scammer Payback with Python, Grok Goes Unhinged, The Future of Chromium and mo

Presenting live web scraping demos at a developer conference
01:57 MIN

Presenting live web scraping demos at a developer conference

Tech with Tim at WeAreDevelopers World Congress 2024

Navigating the complexities of modern web scraping
03:28 MIN

Navigating the complexities of modern web scraping

How to scrape modern websites to feed AI agents

Automating reactive performance testing in your CI/CD pipeline
11:21 MIN

Automating reactive performance testing in your CI/CD pipeline

Frontend Performance Testing in practice

Reusing testing logic and integrating with frameworks
03:36 MIN

Reusing testing logic and integrating with frameworks

The Why and How of WebMCP - Alex Nahas

Practical developer tips for automation and presentation
08:36 MIN

Practical developer tips for automation and presentation

WeAreDevelopers LIVE – AI vs the Web & AI in Browsers

Applying automation across multiple platforms
10:33 MIN

Applying automation across multiple platforms

How to achieve web automation with UiPath

Key takeaways for automated performance testing
02:44 MIN

Key takeaways for automated performance testing

Measure and improve frontend performance by using test automation

Featured Partners

Related Videos

See all videos
1, 2, 3... Fastify!
42:08

1, 2, 3... Fastify!

Matteo Collina

Boost Productivity with AI: Figma & Playwright MCP Workflows - Aris Markogiannakis
45:48

Boost Productivity with AI: Figma & Playwright MCP Workflows - Aris Markogiannakis

One-click-to-production: Test and automate your application
31:15

One-click-to-production: Test and automate your application

Bartosz Pietrucha

Frontend Performance Testing in practice
36:50

Frontend Performance Testing in practice

Jonas Kröger

Measure and improve frontend performance by using test automation
25:38

Measure and improve frontend performance by using test automation

Ramona Schwering

From clicks to cribs - How to find your dream home with web scraping
26:08

From clicks to cribs - How to find your dream home with web scraping

Alexander Lichter

The 2025 State of JavaScript Testing
28:33

The 2025 State of JavaScript Testing

Daniel Afonso

Things I learned while writing high-performance JavaScript applications
27:27

Things I learned while writing high-performance JavaScript applications

Michele Riva

Related Articles

View all articles
Daniel Cranney, Chris Heilmann
Dev Digest 215: Agent Memory, JS2026, Googlebot Analysis & Canvas❤️HTML
Inside last week’s Dev Digest 215 . 🗿 Make AI talk like a caveman 🧠 A guide to context engineering for LLMs 🤖 Simon Willison on agentic engineering 🔐 Axios supply chain attack post mortem 🛡️ Designing AI agents to resist prompt injection 🎨 HTML in c...
Dev Digest 215: Agent Memory, JS2026, Googlebot Analysis & Canvas❤️HTML

From learning to earning

Jobs that call for the skills explored in this talk.