NZGames.com Forums
Register FAQ Calendar Mark Forums Read

Go Back   NZGames.com Forums > General > Technology & Troubleshooting
User Name
Password

Reply
 
Thread Tools
Old 16th September 2019, 15:52     #1
MadMax
Stuff
 
powershell - website scraping

If I run the following:


$nzpost = invoke-webrequest -uri https://nzpost.co.nz/tools/address-p...inder/address/[somenumber]/1 long street highbury palmerston north rd1
($nzpost.inputfields.value -split '\n')[1]


... i get the resolved address including the postcode (assuming a valid address).

If i had a csv file like the following:
address1, address2, city, rural
1 long street, highbury, palmerston north, rd1
1 short road, awapuni, palmerston north, rd1

(eg, $csvfile = import-csv -path 'c:\addresses.csv'

How would i do a scrape for-each of the csv file lines?

[somenumber] seems to be a static random that's re-usable.
__________________
My degree of sarcasm depends on your degree of stupidity.

Last edited by MadMax : 16th September 2019 at 15:57.
  Reply With Quote
Old 16th September 2019, 17:12     #2
crocos
 
https://sharepoint.fpweb.net/sharepo...ith-csv-files/

has exactly what you're after. I googled for "powershell csv file"
__________________
Ξ √ Ω L U T ↑ ☼ N

وكل يوم كنت تعيش في العبودية
  Reply With Quote
Old 16th September 2019, 17:14     #3
crocos
 
Or a slightly different way of doing the same:
https://stackoverflow.com/questions/...esponding-data
__________________
Ξ √ Ω L U T ↑ ☼ N

وكل يوم كنت تعيش في العبودية
  Reply With Quote
Old 16th September 2019, 19:01     #4
MadMax
Stuff
 
I'm ok with the handling of the csv file (i believe), just not the syntax on how to call the invoke-webrequest of the for-each.

for example

Import-Csv -Path 'C:\file.csv' | foreach {
-join ($_.Address3,' ', $_.Suburb,' ',$_.Town,' ',$_.Rural )
}

outputs the query i want and i want to run that against
invoke-webrequest -uri https://www.nzpost.co.nz/tools/addre...inder/address/[number]/[{(join query here)}]

I'm trying to work out that [{( bit
__________________
My degree of sarcasm depends on your degree of stupidity.

Last edited by MadMax : 16th September 2019 at 19:02.
  Reply With Quote
Old 16th September 2019, 19:17     #5
MadMax
Stuff
 
right, so:

$query = invoke-webrequest -uri "https://www.nzpost.co.nz/tools/address-postcode-finder/address/[number]/1 long street palmerston north"
($query.inputfields.value -split '\n')[1]

can be changed to:

((invoke-webrequest -uri "https://www.nzpost.co.nz/tools/address-postcode-finder/address/[number]/1 long street palmerston north").inputfields.value -split '\n')[1]
__________________
My degree of sarcasm depends on your degree of stupidity.
  Reply With Quote
Old 16th September 2019, 19:35     #6
MadMax
Stuff
 
so i got the query working but turns out that number isn't static and is based on the address.

poo.

https://www.nzpost.co.nz/tools/addre...inder/address/[number]1 crater lake the moon

will always return the same result based on the number
__________________
My degree of sarcasm depends on your degree of stupidity.
  Reply With Quote
Old 16th September 2019, 20:22     #7
crocos
 
You'll need to register for NZPost's API service

https://www.nzpost.co.nz/business/developer-centre

For non commercial it may even be free, I don't know.
__________________
Ξ √ Ω L U T ↑ ☼ N

وكل يوم كنت تعيش في العبودية
  Reply With Quote
Old 17th September 2019, 09:59     #8
blynk
 
Quote:
Originally Posted by MadMax
so i got the query working but turns out that number isn't static and is based on the address.

poo.

https://www.nzpost.co.nz/tools/addre...inder/address/[number]1 crater lake the moon

will always return the same result based on the number
I was going to say I was surprised you could just scrap that.
  Reply With Quote
Reply


Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump



© Copyright NZGames.com 1996-2023
Site paid for by members (love you guys)