psasync Module: Multithreaded PowerShell

The series I did on Concurrency in PowerShell contains by far the most popular posts on my blog (getting tens of views every day!), particularly the one on using runspaces. The code demonstrated in the post was an early attempt at using the method. Not long after I refactored the functions I was using into something more modular and flexible. Thus, the psasync module was born. It has proved to be a very simple way for me to kick off background work in runspaces in scripts or from the shell. It still has room for improvement (the functions don’t process pipeline input, for instance), but it’s a start. Since I hope to be updating it as suggestions come in or as I find ways to make it more robust, I’ve started a Codeplex project (my first, so be gentle). If you would like to be a contributor feel free to send me something and, if it’s good, I’ll give you credit on the project.

The module contains the following functions:

Get-RunspacePool

This should have, perhaps, been named Create-RunspacePool. As its name suggests, it returns a pool of runspaces that can be used (and reused).

Invoke-Async

This is a much improved version of the function introduced in the initial post. The big improvement was the addition of the AsyncPipeline class definition, which allows the creation of a simple object to keep track of both the pipeline and the AsyncResult handle which is returned by BeginInvoke(). This allows the process of looking at statuses of running processes and consuming results to be much simpler. The function also allows passing an array of parameters for script blocks with multiple arguments.

Receive-AsyncResults

This function wraps the code for pulling results (or errors, as the case may be) off the pipelines in the runspace pool utilizing the AsyncPipeline objects output from Invoke-Async.

Receive-AsyncStatus

A handy function for working with runspaces from the shell, Receive-AsyncStatus simply returns information about the status of the operations running in the pipelines you have invoked. Since Receive-AsyncResults is synchronous, this allows you to continue to work until your last process completes or selectively use Receive-AsyncResults on those that have completed.

Example Code

To demonstrate the use of the module, consider the same scenario presented in the Concurrency series: You have a series of Excel documents that you need to load into an SQL Server database. As before, set up the script block that will execute the real work.

Import-Module psasync

$AsyncPipelines = @()

$ScriptBlock = `
{
    Param($File)
    
    . <your_path>\Import-ExcelToSQLServer.ps1
    
    Import-ExcelToSQLServer -ServerName 'localhost' -DatabaseName 'SQLSaturday' -SheetName "SQLSaturday_1" `
        -TableName $($File.BaseName) -FilePath $($File.FullName)
}

# Create a pool of 3 runspaces
$pool = Get-RunspacePool 3

$files = Get-ChildItem <path-to-files> 

foreach($file in $files)
{
	 $AsyncPipelines += Invoke-Async -RunspacePool $pool -ScriptBlock $ScriptBlock -Parameters $file
}

Receive-AsyncStatus -Pipelines $AsyncPipelines

Receive-AsyncResults -Pipelines $AsyncPipelines -ShowProgress

You’ll notice there is nothing particularly complex here in the code. But, all the warnings from the runspace post apply. Multithreading is awesome and powerful, but use it with care.

About these ads

10 Responses to psasync Module: Multithreaded PowerShell

  1. Pingback: Concurrency in PowerShell: Multi-threading with Runspaces «

  2. Pingback: Multithreading Powershell Scripts « The Surly Admin

  3. Arlo says:

    Hi, your work looks great. Can you help me with this? The below script gets hostnames from hosts.txt but only shows output from 1 system. What am I doing wrong? Finally, where would an export-csv go to catch returned data from all the hosts, preferably in one file? Thanks a ton if you have the time!

    $ScriptBlock = { Param($ComputerName) Get-service -ComputerName $ComputerName | select machinename, name }

    $pool = Get-RunspacePool 3

    $computernames = Get-Content C:\temp\hosts.txt

    foreach($computername in $computernames)
    {
    $AsyncPipelines += Invoke-Async -RunspacePool $pool -ScriptBlock $ScriptBlock -Parameters $computername
    }

    Receive-AsyncStatus -Pipelines $AsyncPipeplines

    Receive-AsyncResults -Pipelines $AsyncPipelines -ShowProgress

    | export-csv C:\tempz\threading2.csv -NoTypeInformation

    • jboulineau says:

      Hi Arlo,

      I noticed three things in your script. One is that you need to add this $AsyncPipelines = @() so that the variable is created as an array. This will make sure the += operator functions properly. Also, in this line Receive-AsyncStatus -Pipelines $AsyncPipeplines the $AsyncPipeplines variable is misspelled. Finally, I think your path for the .csv output file should be c:\temp not c:\tempz.

      Once I fixed those problems I tested your code and it ran perfectly.

  4. Pingback: PowerShell Studio Review |

  5. Great !! Can I use using Powershell REmoting?

  6. Another samples about it?

  7. Nick Duckstein says:

    Would somebody give an example of how to pass multiple parameters to the script.

    Thanks,

    Nick

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: