
Managing automation processes, services, and servers with PowerShell is a fundamental capability for any IT and cybersecurity professional. PowerShell, developed by Microsoft, is a powerful scripting and automation tool that allows you to monitor and manage processes and services on computers and servers. In this article, we look at how to use PowerShell to monitor, start, stop, and configure processes and services, as well as to manage servers and their configuration. You will learn to use a variety of commands and scripts to effectively manage your IT infrastructure.
This article will also provide you with information on creating scripts to automate tasks such as regularly updating servers, monitoring system health, and fixing errors. There are a huge number of useful services on the Internet that you can work with by accessing their resources using the NTTR protocol. Web developers work with these HTTR requests all the time to access external API functions or to test their own applications. Traditionally, this is done using special applications (for example, Postman), console utilities such as cURL and wget, as well as NTTR clients integrated into development environments. PowerShell offers two standard cmdlets for accessing web resources using the NTTR protocol: Invoke-WebRequest and Invoke-RestMethod.
You can use the Invoke-WebRequest cmdlet to send a request to a web server and get a response from it.
The Invoke-WebRequest cmdlet works well for Tayuke parsing: this cmdlet can save pages to local disk, it is similar to the wget console utility, and even has the following alias:
PS C:\Script> Get-Alias wget CommandType Name Alias wget -> Invoke-WebRequest
Use this cmdlet to navigate to a simple page, such as Domain Example (https://example.com/index.html) (see Figure 1). 14.1
By default, Invoke-WebRequest makes an HTTP GET request to a resource on the web server, with the resource address specified as the value of the -Uri parameter. As a result, an object of type Htm1WebResponseObject is returned, which stores information about the server’s response:
PS C:\Script> $web = Invoke-WebRequest -Uri https://exanple.com/index.ht3nl PS C:\Script> $web | Get-Member TypeName: Microsoft.PowerShell.Commands.HtmlWebResponseObject PS C:\Script> $web StatusCode : 200 StatusDescription : OK Content : <!doctype html> <html> <head> <title>Example Domain</title> <meta charset="utf-8” /> <meta http-equiv="Content-type" content="text/html; char set=utf-8” /> <meta name="viewport" conten... RawContent : HTTP/1.1 200 OK Age: 497890 Vary: Accept-Encoding X-Cache: HIT Forms Content-Length: 1256 Cache-Control: max-age=604800 Content-Type: text/html; charset=UTF-8 Date: Mon, 12 Jul 2021 16:05:14 GMT Exp... : {} Headers : {[Age, 497890], [Vary, Accept-Encoding], [X-Cache, HIT], [Content-Length, 1256]... } Images InputFields Links : {} : {} : {@{innerHTML=More information...; innerText=More information ...; outerHTML=<A href="https://www.iana.org/domains/example ">More information...</A>; outerText=More information...; tagName=A; href=https://www.iana.org/domains/example}} ParsedHtml RawContentLength : mshtml.HTMLDocumentClass : 1256
The statuscode field contains the response code from the server (200 for our example), the statusDescription field contains a text description of the response (OK).
The content of the response from the server is stored as a string in the content field. In our case, the HTML code will be written here:
PS C:\Script> $web.Content <!doctype html> <html> <head> <title>Example Domain</title> <meta charset="utf-8" /> <meta http-equiv="Content-type" content="text/html; charset=utf-8" /> <meta name-'viewport” content="width=device-width, initial-scale=l" /> <style type="text/css"> body { background-color: #f0f0f2; margin: 0; padding: 0; font-family: -apple-system, system-ui, BlinkMacSystemFont, "Segoe UI", "Open Sans", "Helvetica Neue", Helvetica, Arial, sans-serif; } div { width: 600px; margin: 5em auto; padding: 2em; background-color: #fdfdff; border-radius: 0.5em; box-shadow: 2px 3px 7px 2px rgba(0,0,0,0.02); } a:link, a:visited { color: #38488f; text-decoration: none; @media (max-width: 700рх) { div { margin: 0 auto; width: auto; } } </style> </head> <body> <div> <hl>Example Domain</hl> <p>This domain is for use in illustrative examples in documents. You may use this domain in literature without prior coordination or asking for permission.</p> <p><a href=”https://www.iana.org/domains/example">More information...</a></p> </div> </body> </html>
The Rawcontent field records the complete response from the server with HTTP headers at the beginning:
PS C:\Script> $web.RawContent НТТР/1.1 200 OK Age: 497890 Vary: Accept-Encoding X-Cache: HIT Content-Length: 1256 Cache-Control: max-age=604800 Content-Type: text/html; charset=UTF-8 Date: Mon, 12 Jul 2021 16:05:14 GMT Expires: Mon, 19 Jul 2021 16:05:14 GMT ETag: "3147526947+ident" Last-Modified: Thu, 17 Oct 2019 07:18:26 GMT Server: ECS (dcb/7F83) <!doctype html> <html> <head> <title>Example Domain</title>
Also, the response headers are separated and stored in the form of a hash table as headers:
PS C:\Script> $web.headers Key Value Age 497890 Vary Accept-Encoding X-Cache HIT Content-Length 1256 Cache-Control max-, Content-Type Date Expires ETag Last-Modified Server max-age=604800 text/html; charset=UTF-8 Mon, 12 Jul 2021 16:05:14 GMT Mon, 19 Jul 2021 16:05:14 GMT "3147526947+ident" Thu, 17 Oct 2019 07:18:26 GMT ECS (dcb/7F83)
To save the response from the server as a local file, use the -OutFile KEY when calling Invoke-WebRequest and specify the path to the desired file.
PS C:\Script> Invoke-WebRequest -Uri https://exanple.com/index.html -OutFile page.html
Let’s check the contents of the page.html file and make sure that it contains the HTML markup of the saved page:
PS C:\Script> type .\page.html <!doctype html> <html> <head> <title>Example Domain</title> <meta charset="utf-8" /> <meta http-equiv="Content-type" content=”text/html; charset=utf-8" /> <meta name="viewport" content="width=device-width, initial-scale=l" /> <style type="text/css"> body { background-color: #f0f0f2; margin: 0; padding: 0; font-family: -apple-system, system-ui, BlinkMacSystemFont, "Segoe UI", "Open Sans", "Helvetica Neue", Helvetica, Arial, sans-serif; div { width: 600px; margin: 5em auto; padding: 2em; background-color: ttfdfdff; border-radius: 0.5em; box-shadow: 2px 3px 7px 2px rgba(0,0,0,0.02); a:link, a:visited { color: #38488f; text-decoration: none; } @media (max-width: 700px) { div { margin: 0 auto; width: auto; } } </style> </head> <body> <div> <hl>Example Domain</hl> <p>This domain is for use in illustrative examples in documents. You may use this domain in literature without prior coordination or asking for permission.</p> <p><a href="https://www.iana.оrg/domains/example”>More information... </ax/p> </div> </body> </html>
In a similar way, it is possible to store not only HTML files, but also other types of resources (text, graphic or multimedia files, etc.) that can be referred to the site.
The Forms (forms), Images (images), InputFields (INPUT FIELDS) and Links (links) properties of the HtmiwebResponseObject object contain arrays of objects that describe the corresponding elements of the HTML markup received from the server. By processing these collections, we can get information about the items we are interested in on the loaded page.
We will highlight all links to the habr.com site on this page. To do this, you need to filter the $web array. Links, leaving in it objects whose value of the href property corresponds to the mask *habr.com*:
PS C:\Script> $habr_links = $web.Links | Where-Object href -like '*habr.com*'
$habr_iinks contains two PSCustomObject objects containing different HTML attributes for the links:
PS C:\Script> $habr_links.count 2 PS C:\Script> $habr_links | Get-Member
For example, let’s output the value:
PS C:\Script> $habr_links | ForEach-Object {$_.href + " - " + $_.innerText } https://habr.com/ru/company/ruvds/blog/487876/ - Что такое Windows PowerShell и с чем его едят? / Хабр https://habr.com/ru/company/ruvds/blog/487876/ - habr.com >ru/company/ruvds/blog/487876/
The ParsedHtml property of the HtmlWebResponseObject object contains an object of type mshtmi.HTMLDocumentciass, which provides access to the DOM tree of the loaded HTML page.
PS С:\Users\andrv> $html = $web. ParsedHtml PS C:\Users\andrv> Get-Member -Inputobject $html TypeName: mshtmi.HTMLDocumentciass
The Invoke-WebRequest cmdlet allows not only to perform GET requests, but also to call other methods defined in the HTTP protocol (delete, head, merge, patch, post, put, trace). And so the desired method is specified as the value of the -Method parameter. Consider an example of executing a request with the HTTP post method, which is often used to transfer data from web forms or upload files to a server. We will contact the resource http://httpbin.org/post, as a result, the server should inform us about the received data. We will pass two parameters with the names name and lastName, which we will place in the hash table $params:
PS С:\Users\andrv> $params = @{name='Andrey'; lastName='Popov'}
Let’s execute Invoke-WebRequest with the post method, placing the parameters passed in the request body (the -Body parameter):
PS С:\Users\andrv> Invoke-WebRequest -Uri http://httpbin.org/post -Method POST -Body $params StatusCode StatusDescription Content : 200 : OK : { "args": {}, "data": "", "files": {}, "form": { "lastName": "Popov", "name": "Andrey" }, "headers": { "Content-Length": "26", "Content-Type": "application/x-www-form.. RawContent : HTTP/1.1 200 OK Connection: keep-alive Access-Control-Allow-Origin: ★ Access-Control-Allow-Credentials: true Content-Length: 503 Content-Type: application/json Date: Fri, 16 Jul 2021 03:03:13 GM... Forms Headers : {} : {[Connection, keep-alive], [Access-Control-Al low-Origin, *], [Access-Control-Allow-Credent ials, true], [Content-Length, 503]...} Images InputFields Links ParsedHtml RawContentLength : {} : {} : {} : mshtml.HTMLDocumentClass : 503
The parameters of the invoke-webRequest cmdlet sent to the server automatically result in the application/x-www-form-urlencoded format used when submitting data from web forms. The response in the content field indicates that the server accepted our parameters and determined that they were sent from the form. Sometimes you need to send data to the server in JSON format, and not in application/x-www-form-urlencoded. In this case, you need to specify the -contentType parameter with the value application/json. Example:
PS С:\Users\andrv> $json_params = "{ 'nameAndrey', ' lastNamePopov' }" PS C:\Users\andrv> Invoke-WebRequest -Uri http://httpbin.org/post -ContentType "application/json" -Method POST -Body $json_params StatusCode : 200 StatusDescription : OK Content : { "args": {}, "data": "{ ’name':'Andrey’, 'lastNamePopov ’ Г, "files": {}, "form": {}, "headers": { "Content-Length": "39", "Content-Type": "application/json", "Host": "... RawContent : HTTP/1.1 200 OK Connection: keep-alive Access-Control-Allow-Origin: * Access-Control-Allow-Credentials: true Content-Length: 475 Content-Type: application/json Date: Fri, 16 Jul 2021 03:43:04 GM... Forms Headers : {} : {[Connection, keep-alive], [Access-Control-Allo w-Origin, *], [Access-Control-Allow-Credentials , true], [Content-Length, 475]...} Images InputFields Links ParsedHtml RawContentLength : И : {} : (} : mshtml.HTMLDocumentClass : 475
As you can see, with this method of sending the request, the server script that processes access to the http://httpbin.org/post resource extracted the received data and, when forming its JSON response, placed it in the data field, not in the form field. The Invoke-RestMethod cmdlet if we are contacting a web service that supports the REST API, the server’s response will most likely contain structured data in JSON or XML format. So, in the previous example, using the Invoke-WebRequest cmdlet, a POST request was sent to the http://httpbin.org/post resource and a JSON string was received in response:
post -Method POST -Body @ {name=' Andrey ’ ; lastName= ’ Popov' } PS C:\Users\andrv> $web.Content { "args”: {}, ’’data”: "files": {}, "form": { "lastName": "Popov", "name": "Andrey" }, "headers": { "Content-Length": "26", "Content-Type": "application/x-www-form-urlencoded", "Host": "httpbin.org", "User-Agent": "Mozilla/5.0 (Windows NT; Windows NT 10.0; ru-RU) WindowsPowerShell/5.1.19041.1023", "X-Amzn-Trace-Id": "Root=l-60f2a42c-18f152db3b76b5d66dl73e31" }, "json": null, "origin": "85.95.179.209", "url": "http://httpbin.org/post"
To work with response fields, we must convert the JSON to a PowerShell object using the convertFrom-Json cmdlet (we covered similar conversions in Chapter 12):
PS С:\Users\andrv> $response = $web.Content | ConvertFrom-Json PS C:\Users\andrv> $response args : data : files : form : @{lastName=Popov;name=Andrey} headers : @{Content-Tength=26; Content-Type=application/x-www-form-urlencoded; Host-httpbin.org; User-Agent=Mozilla/5.0 (Windows NT; Windows NT 10.0; ru-RU) Windows PowerShell/5.1.19041.1023; X-Amzn-Trace-Id=Root=l-60f2a42c-18f152db3b76b5d66dl73e31} j son : origin : 85.95.179.209 url : http://httpbin.org/post PS C:\Users\andrv> $response.form.name Andrey
In such cases, when we receive structured data from the server, it is more convenient to use the invoke-RestMethod cmdlet, which acts similarly to Invoke-WebRequest and has the same parameters, but at the same time automatically converts the response from the server into a PowerShell object. Let’s execute our request using invoke-RestMethod, saving the result in the $result variable:
PS С:\Users\andrv> $result = Invoke-RestMethod -Uri http://httpbin.org/ post -Method POST -Body @ (name='Andrey' ; lastName=' Popov'}
We will check the type and content of the variable $ result:
PS С:\Users\andrv> Get-Member -Inputobject $result TypeName: System.Management.Automation.PSCustomObject uri NoteProperty string url=http://httpbin.org/post PS C:\Users\andrv> $result Name MemberType Definition Equals Method bool Equals(System.Object obj) GetHashCode Method int GetHashCode() GetType Method type GetType() ToString Method string ToString() args NoteProperty System.Management.Automation.PSCusto data NoteProperty string data= files NoteProperty System.Management.Automation.PSCusto form NoteProperty System.Management.Automation.PSCusto headers NoteProperty System.Management.Automation.PSCusto j son NoteProperty object json=null origin NoteProperty string origin=85.95.179.209 args : data : files : form : @(lastName=Popov;name=Andrey} headers : @{Content-Length=26; Content-Type=application/x-www-form-urlencoded; Host=httpbin.org; User-Agent=Mozilla/5.0 (Windows NT; Windows NT 10.0; ru-RU) Windows PowerShell/5.1.19041.1023; X-Amzn-Trace-Id=Root=l-60f2a897-6354ce272644b6d412560a9b} j son : origin : 85.95.179.209 uri : http://httpbin.org/post
As you can see, instead of a string in JSON format, we get a PowerShell object of type System.Management.Automation. PSCustomObject And we can immediately refer to the necessary ones properties:
PS С:\Users\andrv> $result.form.lastName Popov
Therefore, it is more convenient to use the Invoke-webRequest cmdlet to make HTTP requests to web resources that return HTML pages, and the Invoke-RestMethod cmdlet is more suitable for working with external REST APIs that return structured data.
PowerShell has two standard cmdlets for accessing web resources using the HTTP protocol: Invoke-webRequest and Invoke-RestMethod.
These cmdlets work similarly to each other, except that Invoke-RestMethod automatically converts the response from the server into a PowerShell object.