Top 5 – Pure Storage Technical Blog Posts 2014

Today I thought it would be pretty cool to list out my favorite 5 technical blog posts that pertain to Pure Storage. These are posts that I use to show customers how to get things done without re-inventing the wheel. Big thanks to Barkz and Cody for all the hard work they put in this year. Looking forward to even more awesomeness this year.

SQL Server 2014 Prod/Dev with VMware PowerCLI and Pure Storage PowerShell Toolkit – Rob “Barkz” Barker

Enhanced UNMAP script using with PowerCLI and RESTful API – Cody Hosterman

VMware PowerCLI  and Pure Storage – Cody Hosterman
Check out the great script to set all the vSphere Best Practices for the Pure Storage Flash Array.

Pure Storage PowerShell Toolkit Enhancements – Rob “Barkz” Barker

PowerActions – The PowerCLI Plugin for the vSphere Web Client with UNMAP – Cody Hosterman

JO-Unicorn-Rainbow

VMware vCenter Operations Manager and Pure Storage Rest API

I was playing with the REST API and Powershell in order to provision vSphere Datastores. I started to think what else could we do with all the cool information we get from the Pure Storage REST API?
I remembered some really cool people here and here had used the open HTTP Post adapter. So I started to work on how to pull data out of the Flash Array and into vCOPS.

Pure Dashboard

media_1407352996822.png

We already get some pretty awesome stats in the Pure web GUI. What we don’t get is the trends and analysis. Also I don’t see how my data reduction increases and decreases over time. Also I don’t get stats from multiple arrays.

First Dashboard with Array Stats, Heat Map, and Health based in vCops Baseline

media_1407353219054.png
media_1407360492500.png

Array Level Stats

First each of these scripts require Powershell 4.0.
1. Enter the Flash Array Names in the variable for $FlashArrayName. You can see I have 4 arrays in the Pure SE Lab.
2. I create a file with the credential to vCOPS. Since we are going to schedule this script to run every few minutes you need to create this file. More information on creating that credential here http://blogs.technet.com/b/robcost/archive/2008/05/01/powershell-tip-storing-and-using-password-credentials.aspx

You MUST read and do that to create the cred.txt file in c:\temp that I reference in the script.

3. Change the $url variable to be the IP or name of your vCOPS UI server.
4. Don’t forget to modify the Pure Flash Array and Password in each script.

Find it on GitHub https://github.com/2vcps/purevcops-array

[code]
cls
[System.Net.ServicePointManager]::ServerCertificateValidationCallback = { $true }
$FlashArrayName = @(‘pure1′,’pure2′,’pure3′,’pure4’)

$AuthAction = @{
password = "pass"
username = "user"
}

# will ignore SSL or TLS warnings when connecting to the site
[System.Net.ServicePointManager]::ServerCertificateValidationCallback = {$true}
$pass = cat C:\temp\cred.txt | ConvertTo-SecureString
$mycred = New-Object -TypeName System.Management.Automation.PSCredential -argumentlist "admin",$pass

# function to perform the HTTP Post web request
function post-vcops ($custval,$custval2,$custval3)
{
# url for the vCOps UI VM. Should be the IP, NETBIOS name or FQDN
$url = "<vcops ip>"
#write-host "Enter in the admin account for vCenter Operations"

# prompts for admin credentials for vCOps. If running as scheduled task replace with static credentials
$cred = $mycred

# sets resource name
$resname = $custval3

# sets adapter kind
$adaptkind = "Http Post"
$reskind = "Pure FlashArray"

# sets resource description
$resdesc = "<flasharraydesc>"

# sets the metric name
$metname = $custval2

# sets the alarm level
$alrmlev = "0"

# sets the alarm message
$alrmmsg = "alarm message"

# sets the time in epoch and in milliseconds
#This is setting us 7 hours behind
$epoch = [decimal]::Round((New-TimeSpan -Start (get-date -date "01/01/1970") -End (get-date)).TotalMilliseconds)

# takes the above values and combines them to set the body for the Http Post request
# these are comma separated and because they are positional, extra commas exist as place holders for
# parameters we didn’t specify
$body = "$resname,$adaptkind,$reskind,,$resdesc`n$metname,$alrmlev,$alrmmsg,$epoch,$custval"

# executes the Http Post Request
Invoke-WebRequest -Uri "https://$url/HttpPostAdapter/OpenAPIServlet" -Credential $cred -Method Post -Body $body
#write-host $resname
#write-host $custval2 "=" $custval "on" $custval3
}
ForEach($element in $FlashArrayName)
{
$faName = $element.ToString()
$ApiToken = Invoke-RestMethod -Method Post -Uri "https://${faName}/api/1.1/auth/apitoken" -Body $AuthAction

$SessionAction = @{
api_token = $ApiToken.api_token
}
Invoke-RestMethod -Method Post -Uri "https://${faName}/api/1.1/auth/session" -Body $SessionAction -SessionVariable Session

$PureStats = Invoke-RestMethod -Method Get -Uri "https://${faName}/api/1.1/array?action=monitor" -WebSession $Session
$PureArray = Invoke-RestMethod -Method Get -Uri "https://${faName}/api/1.1/array?space=true" -WebSession $Session
ForEach($FlashArray in $PureStats) {

$wIOs = $FlashArray.writes_per_sec
$rIOs = $FlashArray.reads_per_sec
$rLatency = $FlashArray.usec_per_read_op
$wLatency = $FlashArray.usec_per_write_op
$queueDepth = $FlashArray.queue_depth
$bwInbound = $FlashArray.input_per_sec
$bwOutbound = $FlashArray.output_per_sec
}
ForEach($FlashArray in $PureArray) {

$arrayCap =($FlashArray.capacity)
$arrayDR =($FlashArray.data_reduction)
$arraySS =($FlashArray.shared_space)
$arraySnap =($FlashArray.snapshots)
$arraySys =($FlashArray.system)
$arrayTP =($FlashArray.thin_provisioning)
$arrayTot =($FlashArray.total)
$arrayTR =($FlashArray.total_reduction)
$arrayVol =($FlashArray.volumes)
}

post-vcops($wIOs)("Write IO")($faName)
post-vcops($rIOs)("Read IO")($faName)
post-vcops($rLatency)("Read Latency")($faName)
post-vcops($wLatency)("Write Latency")($faName)
post-vcops($queueDepth)("Queue Depth")($faName)
post-vcops($bwInbound)("Input per Sec")($faName)
post-vcops($bwOutbound)("Output per Sec")($faName)

post-vcops($FlashArray.capacity)("Capacity")($faName)
post-vcops($FlashArray.data_reduction)("Real Data Reduction")($faName)
post-vcops($FlashArray.shared_space)("Shared Space")($faName)
post-vcops($FlashArray.snapshots)("Snapshot Space")($faName)
post-vcops($FlashArray.system)("System Space")($faName)
post-vcops($FlashArray.thin_provisioning)("TP Space")($faName)
post-vcops($FlashArray.total)("Total Space")($faName)
post-vcops($FlashArray.total_reduction)("Faker Total Reduction")($faName)
post-vcops($FlashArray.volumes)("Volumes")($faName)

}
[/code]

 

For Volumes

Find it on GitHub https://github.com/2vcps/purevcops-volumes

[code]
cls
[System.Net.ServicePointManager]::ServerCertificateValidationCallback = { $true }
$FlashArrayName = @(‘pure1′,’pure2′,’pure3′,’pure4’)

$AuthAction = @{
password = "pass"
username = "user"
}

# will ignore SSL or TLS warnings when connecting to the site
[System.Net.ServicePointManager]::ServerCertificateValidationCallback = {$true}
$pass = cat C:\temp\cred.txt | ConvertTo-SecureString
$mycred = New-Object -TypeName System.Management.Automation.PSCredential -argumentlist "admin",$pass

# function to perform the HTTP Post web request
function post-vcops ($custval,$custval2,$custval3,$custval4)
{
# url for the vCOps UI VM. Should be the IP, NETBIOS name or FQDN
$url = "<vcops ip or name>"
#write-host "Enter in the admin account for vCenter Operations"

# prompts for admin credentials for vCOps. If running as scheduled task replace with static credentials
$cred = $mycred

# sets resource name
$resname = $custval

# sets adapter kind
$adaptkind = "Http Post"
$reskind = "Flash Volumes"

# sets resource description
$resdesc = $custval4

# sets the metric name
$metname = $custval2

# sets the alarm level
$alrmlev = "0"

# sets the alarm message
$alrmmsg = "alarm message"

# sets the time in epoch and in milliseconds
#This is setting us 7 hours behind
$epoch = [decimal]::Round((New-TimeSpan -Start (get-date -date "01/01/1970") -End (get-date)).TotalMilliseconds)

# takes the above values and combines them to set the body for the Http Post request
# these are comma separated and because they are positional, extra commas exist as place holders for
# parameters we didn’t specify
$body = "$resname,$adaptkind,$reskind,,$resdesc`n$metname,$alrmlev,$alrmmsg,$epoch,$custval3"

# executes the Http Post Request
Invoke-WebRequest -Uri "https://$url/HttpPostAdapter/OpenAPIServlet" -Credential $cred -Method Post -Body $body

write-host $custval,$custval2,$custval3
}
ForEach($element in $FlashArrayName)
{
$faName = $element.ToString()
$ApiToken = Invoke-RestMethod -Method Post -Uri "https://${faName}/api/1.1/auth/apitoken" -Body $AuthAction

$SessionAction = @{
api_token = $ApiToken.api_token
}
Invoke-RestMethod -Method Post -Uri "https://${faName}/api/1.1/auth/session" -Body $SessionAction -SessionVariable Session

$PureStats = Invoke-RestMethod -Method Get -Uri "https://${faName}/api/1.1/array?action=monitor" -WebSession $Session
$PureVolStats = Invoke-RestMethod -Method Get -Uri "https://${faName}/api/1.1/volume?space=true" -WebSession $Session
ForEach($Volume in $PureVolStats) {
#$Volume.data_reduction
#$Volume.name
#$Volume.volumes
#$Volume.shared_space
#$Volume.system
#$Volume.total
#$Volume.total_reduction
#$Volume.snapshots
$adjVolumeSize = ($Volume.Size /1024)/1024/1024
#$Volume.thin_provisioning

post-vcops($Volume.Name)("Volume Size")($adjVolumeSize)($faName)
post-vcops($Volume.Name)("Volume Data Reduction")($Volume.data_reduction)($faName)
post-vcops($Volume.Name)("Volumes")($Volume.volumes)($faName)
post-vcops($Volume.Name)("Shared Space")($Volume.shared_space)($faName)
post-vcops($Volume.Name)("System")($Volume.system)($faName)
post-vcops($Volume.Name)("Total")($Volume.total)($faName)
post-vcops($Volume.Name)("Total Reduction")($Volume.total_reduction)($faName)
post-vcops($Volume.Name)("Thin Provisioning")($Volume.thin_provisioning)($faName)
post-vcops($Volume.Name)("Snapshots")($Volume.snapshots)($faName)
}
}
[/code]

Once each of the scripts is working schedule them as a task on a windows server. I do one for volumes and one for arrays and run them every 5 minutes indefintely. This will start to dump the data into vCOPS.

Now you can make Dashboards.

Creating Dashboards

media_1408382785427.png

Login to the UI for vCOPS. You must by in the custom UI, the standar UI hides all of the cool non-vSphere customization you can do.

 

Go to Environment –> Environment Overview

media_1408383055768.png

Expand Resource Kinds

media_1408383114822.png

This lets you know that data is being accepted to the array. Other than the Powershell script bombing out and failing this is the only way you know it is working. Now for a new Dashboard.

Click Dashboards -> Add

media_1408383203181.png

Drag Resources, Metric Selector, Metric Graph and Heat Map to the Right

media_1408383262000.png

Name it and Click OK

Adjust the Layout

media_1408383477679.png

I like a nice Column for information and a bigger display area for graphs and heat maps. Adjust to your preference.

Edit the Resources Widget

media_1408383579549.png

Edit the Name and filters to tag

media_1408383667269.png

Now we just see the Flash Arrays

media_1408383734620.png
media_1408383840440.png

Select your Resource Provider I named mine Lab Flash Arrays as the Providing Widget for the Metric Selector. Also Select the Lab Flash Arrays and Metric Selector as the Providing Widgets for the Metric Graph.

Edit the Metric Graph Widget by clicking the gear icon

media_1408384372245.png

I change the Res. Interaction Mode to SampleCustomViews.xml. This way when I select a Flash Array the Graph does show up until I double click the Metric in the Metric Selector. You are of course free to do it as you like.

The Heat Map

media_1408384493307.png

Edit the heat map and you will find tons of options.

media_1408384631976.png

Create a Configuration

media_1408384728117.png

Name the New Configuration

media_1408384811714.png

Group by and Resource Kinds

media_1408384843862.png

Group by the Resource Kind and then select Pure Flash Array in the drop down.

Select the Metric to Size the Heatmap and Color the Heatmap

media_1408384873077.png

Adjust the colors if you think Read and Green are boring

media_1408384896168.png

Save the Config!

media_1408384924548.png

Look! A cool new heatmap

media_1408384959172.png

Do this for all the metrics you want to have as a drop down in teh dashboard.

Obviously there are a lot more things you can do with the Dashboards and widgets. Hopefully this is enough to get you kicked off.

A Brand New Dashboard

media_1408385301227.png

Provision vSphere Datastores on Pure Storage Volumes with Powershell

A week or so ago our Pure Storage powershell guru Barks @themsftdude sent out some examples of using Powershell to get information via the Pure Storage REST API. My brain immediately started to think how we could combine this with PowerCLI to get a script to create the LUN on Pure and then the datastore on vSphere. So now provision away with Powershell! You know, if that is what you like to do. We also have a vCenter plugin if you like that better.

So now you can take this code and put it into a file New-PSDataStore.ps1

What we are doing:

1. Login to vCenter and the REST API for the Array.
2. Create the Volume on the Flash Array.
3. Place the new volume in the Hostgroup with your ESX cluster.
4. Rescan the host.
5. Create the new Datastore.

Required parameters:

-FlashArray The name of your array
-vCenter Name of your vCenter host
-vCluster Name of the cluster your hosts are in. If you don’t have clusters (what?) you will need to modify the script slightly.
-HostGroup The name of the hostgroup in the Pure Flash Array.
-VolumeName Name of the volume and datastore
-VolumeSize  Size of the volume. This requires denoting the G for Gigabytes or T or Terabytes
-pureUser The Pure FlashArray username
-pureUser The Pure FlashArray  password

[powershell]
# example usage
#.\new-PSdatastore.ps1 -FlashArray "Array" -vCenter "vcenter" -vCluster "clustername" -HostGroup "HostGroup" -VolumeName "NewVol" -VolumeSize 500G -pureUser pureuser -purePass purepass
#On the Volume Size parameter you must include the letter after the number I have tested <number>G for Gigabytes and <number>T for Terabytes
#Special thanks to Barkz www.themicrosoftdude.com @themsftdude for the kickstart on the API calls.
#Find me @jon_2vcps on the twitters. Please make this script better.
# If you do not have a stored PowerCLI credential you will be prompted for the vCenter credentials.
#Not an official supported Pure Storage product, use as you wish at your own risk.
#

Param(
[Parameter(Mandatory=$true)]
[ValidateNotNullOrEmpty()]
[string] $FlashArray,
[string] $VCenter,
[string] $vCluster,
[string] $HostGroup,
[string] $VolumeName,
[string] $VolumeSize,
[string] $pureUser,
[string] $purePass

)

Add-PSSnapin VMware.VimAutomation.Core

#cls
$vname=$VolumeName
$vSize=$VolumeSize
[System.Net.ServicePointManager]::ServerCertificateValidationCallback = { $true }
$FlashArrayName = $FlashArray
$vCenterServer = $VCenter
$esxHostGroup = $HostGroup
Connect-viserver -Server $vCenterServer

$workHost = get-vmhost -Location $vCluster | select-object -First 1

$AuthAction = @{
password = $purePass
username = $pureUser
}
$ApiToken = Invoke-RestMethod -Method Post -Uri "https://${FlashArrayName}/api/1.1/auth/apitoken" -Body $AuthAction

$SessionAction = @{
api_token = $ApiToken.api_token
}
Invoke-RestMethod -Method Post -Uri "https://${FlashArrayName}/api/1.1/auth/session" -Body $SessionAction -SessionVariable Session

Invoke-RestMethod -Method POST -Uri "https://${FlashArrayName}/api/1.1/volume/${vname}?size=${vSize}" -WebSession $Session
Invoke-RestMethod -Method POST -Uri "https://${FlashArrayName}/api/1.1/hgroup/${esxHostGroup}/volume/${vname}" -WebSession $Session
$volDetails = Invoke-RestMethod -Method GET -Uri "https://${FlashArrayName}/api/1.1/volume/${vname}" -WebSession $Session
$rescanHost = $workHost | Get-VMhostStorage -RescanAllHba
$volNAA = $volDetails.serial
$volNAA = $volNAA.substring(15)
$afterLUN = $workHost | Get-scsilun -CanonicalName "naa.624*${volNAA}"
New-Datastore -VMhost $workHost -Name $vname -Path $afterLUN -VMFS
[/powershell]