Page tree
Skip to end of metadata
Go to start of metadata

This page describes how to setup PowerShell on Windows to support REEF builds.

Java

Make sure Java Development Kit 7 or 8 is installed and $JAVA_HOME points to its installation. Add the following to your PowerShell profile:

 

PowerShell Profile Snippet for Java
# Java aliases
if($Env:JAVA_HOME){
    Set-Alias javac $Env:JAVA_HOME\bin\javac.exe
    Set-Alias java $Env:JAVA_HOME\bin\java.exe
    Set-Alias jar $Env:JAVA_HOME\bin\jar.exe
    Set-Alias jps $Env:JAVA_HOME\bin\jps.exe
} else {
    Write-Host "JAVA_HOME is not set in the environment, verify Java installation and set JAVA_HOME" -ForegroundColor Red
}

Maven

Make sure Maven 3 is installed and $M2_HOME points to the installation. Add following to your PowerShell profile:

PowerShell Profile Snippet for Maven
# Setup maven
if($Env:M2_HOME){
    # Fast builds with no tests, but multithreading
    function mvn-mt{
        $cmd = "`'$Env:M2_HOME\bin\mvn.cmd`' -TC1 -DskipTests $args"
        Invoke-Expression "& $cmd"
    }
    function mvn{
        $cmd = "`'$Env:M2_HOME\bin\mvn.cmd`' $args"
        Invoke-Expression "& $cmd"
    }
} else {
    Write-Host "M2_HOME is not set in the environment, verify Maven installation and set M2_HOME" -ForegroundColor Red
}

Git(Hub)

Install GitHub Desktop. Launch the GUI client once. Launch the GitHub Shell once. Then add the following to your PowerShell profile:

PowerShell Profile Snippet for GitHub
# setup git
if(Test-Path ~\AppData\Local\GitHub){    
    . (Resolve-Path "$env:LOCALAPPDATA\GitHub\shell.ps1")
    . (Resolve-Path "$env:github_posh_git\profile.example.ps1")
}

Managing Pull Requests (Committers only)

As a committer, you frequently need to checkout pull requests in order to check them. Add the following set of functions to your profile to ease that process:

PowerShell Function to checkout pull requests
function REEF-Checkout-PullRequest{
    param
    (
        [Parameter(Position=0, Mandatory=$true, HelpMessage='The pull request number')]
        [int]$PR,
        [Parameter(Mandatory=$false, HelpMessage='The branch to check the PR out into')]
        [string]$B
    )

    $branch = if([string]::IsNullOrEmpty($B)) {"PR-$PR"} else {$B}
    Invoke-Expression "git fetch https://github.com/apache/reef pull/$PR/head:$branch"
    Invoke-Expression "git checkout $branch"
}
PowerShell Function to rebase pull request onto apache master
function REEF-Rebase-PullRequest{
    Invoke-Expression 'git fetch apache; git rebase -i apache/master'
}
PowerShell Function to merge pull request into master
function REEF-Merge-PullRequest{
    param
    (
        [Parameter(Position=0, Mandatory=$true, HelpMessage='The pull request number')]
        [int]$PR
    )
 
    $branch = {"PR-$PR"}
    Invoke-Expression "git checkout master"
    Invoke-Expression "git merge $branch"
}
PowerShell Function to push merged pull request into apache master and your local git repository master
function REEF-Push-PullRequest{
    Invoke-Expression 'git push apache master'
    Invoke-Expression 'git push'
}

With this sequence of functions in your profile, the process of reviewing and merging pull request will look as follows:

Checkout, review and merge pull request
REEF-Checkout-PullRequest N
# ... test pull request ...
REEF-Rebase-PullRequest
# git commit --amend to fix commit message
REEF-Merge-PullRequest N
# last chance to change your mind and not merge!
REEF-Push-PullRequest

Visual Studio

Install Visual Studio 2013, 2015, 2017. Then add the following to your PowerShell profile:

PowerShell Functions to Setup Visual Studio command line environment
# Setup Visual Studio
function Setup-VisualStudio{
    param
    (
        [Parameter(Mandatory=$true, HelpMessage='Version')]
        $Version
    )
    $Script = "vcvarsall.bat&set"
    if ($Version -eq "2013"){
        $VisualStudioPath = 'c:\Program Files (x86)\Microsoft Visual Studio 12.0\VC'
    }elseif ($Version -eq "2015"){
        $VisualStudioPath = 'c:\Program Files (x86)\Microsoft Visual Studio 14.0\VC'
    }elseif ($Version -eq "2017"){
        $VisualStudioPath = 'C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Auxiliary\Build'
        $Script = "vcvarsall.bat x64 & set"
    }else{
        throw "Please provide a Version of Visual Studio: 2013, 2015, 2017"
    }
    if(Test-Path $VisualStudioPath){
        Push-Location $VisualStudioPath
        cmd /c "$Script" |
        foreach {
            if ($_ -match "=") {
                $v = $_.split("=")
                Set-Item -Force -Path "ENV:\$($v[0])"  -Value "$($v[1])"
            }
        }
        Pop-Location
    } else {
        Write-Host "Visual Studio $Version not found at: $VisualStudioPath" -ForegroundColor Red
    }
}

With this in your profile you can start Visual Studio with:

Start Visual Studio
### 2013
Setup-VisualStudio 2013
### 2015
Setup-VisualStudio 2015

Building (and cleaning) REEF

Assuming $REEFSourcePath points to the right folder, the following two functions can be used to build and reliably clean REEF:

Build and Clean Functions
function Build-REEF{
    if($Env:REEFSourcePath){
        Invoke-Expression 'msbuild $Env:REEFSourcePath\lang\cs\Org.Apache.REEF.sln /p:Configuration="Release" /p:Platform="x64" /m'
    } else {
        Write-Host "You must set REEFSourcePath to the root of your enclistment to use Build-REEF command" -ForegroundColor Red
    }
}
function Clean-REEF{
    if($Env:REEFSourcePath){
        Invoke-Expression 'msbuild $Env:REEFSourcePath\lang\cs\Org.Apache.REEF.sln /p:Configuration="Release" /p:Platform="x64" /t:Clean /m'
        Get-ChildItem -Path $Env:REEFSourcePath\lang\cs\ -Recurse -Filter obj                | Remove-Item -Recurse
        Get-ChildItem -Path $Env:REEFSourcePath\lang\cs\ -Recurse -Filter bin                | Remove-Item -Recurse
        Get-ChildItem -Path $Env:REEFSourcePath\lang\cs\ -Recurse -Filter target             | Remove-Item -Recurse
        Get-ChildItem -Path $Env:REEFSourcePath\lang\cs\ -Recurse -Filter REEF_LOCAL_RUNTIME | Remove-Item -Recurse
        Get-ChildItem -Path $Env:REEFSourcePath\lang\cs\ -Recurse -Filter TestResults        | Remove-Item -Recurse
        Get-ChildItem -Path $Env:REEFSourcePath\lang\cs\ -Recurse -Filter packages           | Remove-Item -Recurse
    } else {
        Write-Host "You must set REEFSourcePath to the root of your enlistment to use Clean-REEF command" -ForegroundColor Red
    }
}

Verifying Release artifacts

We compute hashes for the releases using the GNU Core Utils. The functions below can be used to verify those hashes.

Hash Functions for PowerShell
function Verify-Hash{
    param
    (
        [Parameter(Mandatory=$true, HelpMessage='The .MD5 file.')]
        [string]$File,
        [Parameter(Mandatory=$true, HelpMessage='The Algorithm to use')]
        [string]$Algorithm
    ) 
    foreach ($line in (Get-Content $File)) {
        $fields = $line -split '\s+'
        $hash = $fields[0].Trim().ToUpper()
        $filename = $fields[1].Trim()
        if($filename.StartsWith("*")){
            $filename = $filename.Substring(1).Trim()
        }
        $computedHash = (Get-FileHash -Algorithm $Algorithm $filename).Hash.ToUpper()
        if($hash.Equals($computedHash)){
            Write-Host $filename, ": Passed"
        }else{
            Write-Host $filename, ": Not Passed"
            Write-Host "Read from file: ", $hash
            Write-Host "Computed:       ", $computedHash
        }
    }
}
function Verify-MD5{
    param
    (
        [Parameter(Position=0, Mandatory=$true, HelpMessage='The .MD5 file.')]
        [string]$File
    ) 
    Verify-Hash -Algorithm MD5 -File $File
}
function Verify-SHA512{
    param
    (
        [Parameter(Position=0, Mandatory=$true, HelpMessage='The .SHA512 file.')]
        [string]$File
    ) 
    Verify-Hash -Algorithm SHA512 -File $File
}
  • No labels