What and let's go ahead and continue extending our project I know we are going to implement something 2
more fancy and interesting. 3
We're going to download web pages and write them to the desk. 4
So in our system we have different kinds of bookmarks. 5
A bookmark can be a weblink. 6
It can be a movie or it can be a book. 7
Now when it comes to babblings when a user bookmarks a web link we were saving the title of the eyeblink 8
along with your old information. 9
But now we want to extend the project to also download the corresponding web page content. 10
Now that would give us a couple of benefits. 11
One of them is we can maintain a local cache of the web page. 12
So if tomorrow the post is not available if it is taken on our of the web page is not available then 13
we still have the local cache of the web page and the user can still browse the bitch. 14
But the more important benefit is that they can index all those web pages. 15
Now if we index those web pages it means that the user can actually search this business. 16
So indexing basically allows us to provide search capability an index is basically a data structure 17
which allows us to search text in a very efficient way. 18
And there are third party libraries like so lock on Lucine which can help us in indexing their documents 19
. 20
So we're not going to index the web pages but we're going to just download the web page and we're going 21
to write it to their desk. 22
OK. 23
And so there are a couple of pasts involved here. 24
So when we say we are going to download a web page what we are doing is we are actually doing it normally 25
read operation. 26
But in this case the source is in a remote host earlier in the last lesson. 27
We saw that when we read we actually read from a file. 28
So do they are the source of just fine. 29
But now with this extension the source is basically remote host. 30
So it's the network. 31
So once we read the document from the remote host then we need to write it with a disk on product we 32
use the our writer causus. 33
OK so we are going to do that. 34
So let's just go ahead and make these extensions. 35
So I have created this new class called hysterically picnic and it has been added to the Udal package 36
which was created in the last class on the history class. 37
Has this method us don't work and all it is going to do as as the name suggests it's going to download 38
a webpage. 39
So the input to this method is a you are off of a page. 40
You are a lawful eyeblink which is a bookmark and it's going to honor the page and written the contents 41
of the web page as a string. 42
And here we are using some classes from the Jawa dot net package. 43
Since we did not discuss it you don't have to worry about it and it's not even production quality code 44
. 45
So basically here there is this class called hitcher to be your own connection and it represents a connection 46
to the remote host. 47
And then we are invoking this method called Let's get response called on this variable. 48
Can and when we do that if the response code is in between to those 200 and 300 It means that we are 49
able to connect with that. 50
So we're successfully on so we can read the contents of that particular page. 51
OK. 52
And that's what we're going to do here. 53
And so we are in working this matter called Get input stream on the connection object and get input 54
stream simply returns a regular input stream. 55
Ok so now we're going to just read from here. 56
And we know that input stream is the base abstract class for all byte oriented and streams. 57
So let's go ahead and fill this method and it could be part of the I or you do. 58
So let's just say create method and let's just go ahead and fill this matter. 59
So the previous method was reading from a file. 60
And instead of building a string it was writing it into the string every night. 61
So that was the last lesson. 62
So let's just go ahead and copy this because the code is pretty much identical. 63
So the one difference is let's make this as in to keep it simple and file in good stream object is replaced 64
with this input stream. 65
So it's more generic now. 66
And since we want to accumulate the content of the page we want to build a web page content that uses 67
string builder. 68
So we should always use a string builder. 69
I have seen code where people who are concatenating using the plus operation for such an important operation 70
and that's very bad. 71
Now string line is fine let's remove this. 72
To append 73
and we're writing a new line that's because real method is strips new lines lets add some catch glossier 74
to catch clause 75
. 76
So we are dimming that string method we are invoking the true string. 77
It's going to read on the web page content which has been accumulated and this string builder object 78
. 79
So that's about it. 80
And then coding is still UTF 8 and we will be using only you be of 8 in our system because that's like 81
a universal kind of cursor. 82
Now let's go back into history to be correct. 83
And this is fine. 84
Now let's look at the code which is going to invoke this method. 85
So if you're going to view our Java. 86
So we have this browse here and we are 87
basically for a given user we are going to we are doing the bookmarking here and we are going we are 88
invoking the same user bookmark. 89
So that's just going here. 90
It's into the going into book my controller. 91
From here we are going to bookmark manager and have already written some code here. 92
Now we want to download the web page only if the bookmark is an instance of eyeblink. 93
So here we are passing a bookmark which can be a weblink which can be a book or a movie but we want 94
to do this only if it is a weblink. 95
So here we are. 96
Downcasting it took a blink. 97
If it an instance off of a blink and we are working to get you out of the matter because wabbling has 98
a get your delimiter and then we are assigning it to this variable you are in. 99
So it basically has that you are a lot of eyeblink. 100
And we want to download only effect for now if it doesn't end with Dot B-B of which means that if it 101
ends with dockage DML or something like that then we want to download. 102
So then we are connecting in working this download method here. 103
OK. 104
Which is what we haven't seen. 105
And we are passing the Yoran of the page. 106
OK so once we get back the response. 107
So it's assigned to this wearable web page. 108
And if the web page is not equal to none only then we are going to write it to their desk. 109
Sometimes it can be not also for instance if the response code is not within this limit then it means 110
that something is wrong in which case we are going to read on another. 111
OK so let's go back and let's also for the right matter now. 112
OK. 113
So that's also going to get created within the higher your class. 114
So let's go ahead and the input barometer here is the ID because the when we write to the desk we have 115
to create a file and the filename will be the name of the idea. 116
So because the idea is unique to the bookmark we go we can just use that ID. 117
And we are passing the web page content also because that needs to be written to the file. 118
So here it is. 119
So let's just pick this chord here and our are. 120
Now it's very simple. 121
Just keep making it right. 122
So on it as a writer a little bit more explicit. 123
So it's just the exact opposite stuff 124
. 125
OK now let's leave this as UTF 8 and yes we need to give a final name so I'd say Dot hedge DML 126
and so we are again creating a string here from the ID. 127
So we are invoking the value of. 128
And 129
we need to specify the location. 130
So here JRD is my workspace eclipse workspace and truly is our project. 131
And let's create this page a directory called pages and it's right here. 132
So I have already created this directory called pages. 133
So you should also create this page just a directory within that trivial project and all the pages that 134
get downloaded will be read on into that directory. 135
So that's also put the finals last year. 136
So that's what we have here. 137
And we can read more all of this make a writer dark ride and we just 138
do this 139
like Eclipse cleared all the exceptions for us exception catch blocks for us and upset. 140
So 141
OK that's about it. 142
So let's just go ahead and run our application now 143
running something else. 144
And one other small change I'm mean is that in the weblink there was a different you are here for the 145
first two eyeblink but that was taking a lot of time to download. 146
So I just replaced it with something else but later we will change it back but or later when we are 147
dealing with multithreading we will actually see how we can abandon a request if it is taking too long 148
. 149
OK. 150
So right now it's taking like 10 or 15 seconds you don't know that page. 151
So if it is it shouldn't be taking so long. 152
So if something like that happens we should abandon downloading that Biche because that's just not good 153
to download such kind of Lapidus so that we will see later. 154
But this start to just let you know. 155
So I just replace that with this non-IT me quickly launch. 156
So it is downloading downloading what it is downloading it means it's not loading some link. 157
It's done. 158
So let me 159
refresh this on so there are three pages that God created. 160
OK. 161
So you are there are there are three hits the whole pages that go down in order. 162
So that's it. 163
So you just go ahead and I'm going to upload some code into the resources section. 164
So what we have done is I think it's more interesting that we have actually downloaded web pages and 165
we have written them to their desk. 166
So we also know how to write data out to their desk and how to read for them from a network. 167
And that's about it. 168
Thank you. 169
I'm happy coding. 170
And here it is. 171
Here we are saying that we are downloading some songs so that's the next year.