1
00:00:00,030 --> 00:00:00,660
Hey guys,

2
00:00:00,660 --> 00:00:05,190
welcome to Day 48 of 100 Days of Code. Today

3
00:00:05,190 --> 00:00:10,190
we're going to be introducing a new technology for advanced web scraping,

4
00:00:10,890 --> 00:00:13,200
and that is Selenium WebDriver.

5
00:00:13,680 --> 00:00:16,170
Now you might have heard of Selenium WebDriver before

6
00:00:16,200 --> 00:00:21,030
because it's probably one of the most well-known automation and testing tools

7
00:00:21,300 --> 00:00:25,200
for web developers out there. But you might be wondering, well,

8
00:00:25,230 --> 00:00:29,550
we already have Beautiful Soup. So why do we need to learn a new technology?

9
00:00:30,180 --> 00:00:30,450
Well,

10
00:00:30,450 --> 00:00:35,450
one of the things that we've really been limited by is we can't actually use all

11
00:00:35,970 --> 00:00:38,370
the capabilities that browsers can do.

12
00:00:38,580 --> 00:00:42,930
So when we load up a website with beautiful soup, we can't, for example,

13
00:00:42,990 --> 00:00:47,990
type something into the website and then click on something. And to create these

14
00:00:48,120 --> 00:00:49,920
chains of continuous actions

15
00:00:50,280 --> 00:00:55,280
where we basically automate the entire flow of a particular job or a particular

16
00:00:55,950 --> 00:00:58,320
task. To do that,

17
00:00:58,380 --> 00:01:01,230
we're going to need to use Selenium WebDriver.

18
00:01:02,070 --> 00:01:07,070
Now, this is a free tool and it basically allows us to automate the browser,

19
00:01:08,250 --> 00:01:11,520
get the browser to do things automatically

20
00:01:11,760 --> 00:01:16,760
depending on a script or a piece of code that we write. Now,

21
00:01:17,580 --> 00:01:22,580
this is going to enable us to type as well as click as well as scroll.

22
00:01:25,560 --> 00:01:30,560
Basically anything that a human pretty much can do on a website, you can do using

23
00:01:30,930 --> 00:01:33,030
a Selenium driven browser.

24
00:01:33,600 --> 00:01:37,860
It's kind of like we're building a robot and telling it what to do on a browser.

25
00:01:38,310 --> 00:01:43,290
And selenium is the tool that allows the robot to interact and communicate with

26
00:01:43,290 --> 00:01:46,080
the browser. And once we've leveled up,

27
00:01:46,140 --> 00:01:49,290
we'll be able to do all sorts of things like, for example,

28
00:01:49,590 --> 00:01:54,590
play web-based games like cookie clicker automatically to click on this cookie

29
00:01:56,820 --> 00:02:01,820
all by itself using the code instead of us having to manually do it and then to

30
00:02:02,820 --> 00:02:07,820
select upgrades and to purchase them, again, completely automatically.

31
00:02:09,090 --> 00:02:13,530
If you ever wanted to be the best at cookie-clicker or any other web-based

32
00:02:13,530 --> 00:02:18,150
game and you don't want to invest the time in scaling up and getting good at it

33
00:02:18,420 --> 00:02:23,220
yourself, well then this is where automation might help you.

34
00:02:23,940 --> 00:02:25,560
But on a more serious note,

35
00:02:25,590 --> 00:02:29,100
you might use something like selenium to automate

36
00:02:29,100 --> 00:02:34,100
filling in forms or transferring information from an Excel spreadsheet to a

37
00:02:35,190 --> 00:02:36,600
online Google form,

38
00:02:36,630 --> 00:02:40,470
or basically doing anything that is repetitive and tedious,

39
00:02:40,770 --> 00:02:42,390
and you don't want to do it yourself.

40
00:02:43,050 --> 00:02:45,690
So once you are ready, head over to the next lesson

41
00:02:45,900 --> 00:02:49,440
and let's start learning about this new superpower.