DUE TO SPAM, SIGN-UP IS DISABLED. Goto Selfserve wiki signup and request an account.
This page describes how to setup PowerShell on Windows to support REEF builds.
Java
Make sure Java Development Kit 7 or 8 is installed and $JAVA_HOME points to its installation. Add the following to your PowerShell profile:
# Java aliases
if($Env:JAVA_HOME){
Set-Alias javac $Env:JAVA_HOME\bin\javac.exe
Set-Alias java $Env:JAVA_HOME\bin\java.exe
Set-Alias jar $Env:JAVA_HOME\bin\jar.exe
Set-Alias jps $Env:JAVA_HOME\bin\jps.exe
} else {
Write-Host "JAVA_HOME is not set in the environment, verify Java installation and set JAVA_HOME" -ForegroundColor Red
}
Maven
Make sure Maven 3 is installed and $M2_HOME points to the installation. Add following to your PowerShell profile:
# Setup maven
if($Env:M2_HOME){
# Fast builds with no tests, but multithreading
function mvn-mt{
$cmd = "`'$Env:M2_HOME\bin\mvn.cmd`' -TC1 -DskipTests $args"
Invoke-Expression "& $cmd"
}
function mvn{
$cmd = "`'$Env:M2_HOME\bin\mvn.cmd`' $args"
Invoke-Expression "& $cmd"
}
} else {
Write-Host "M2_HOME is not set in the environment, verify Maven installation and set M2_HOME" -ForegroundColor Red
}
Git(Hub)
Install GitHub Desktop. Launch the GUI client once. Launch the GitHub Shell once. Then add the following to your PowerShell profile:
# setup git
if(Test-Path ~\AppData\Local\GitHub){
. (Resolve-Path "$env:LOCALAPPDATA\GitHub\shell.ps1")
. (Resolve-Path "$env:github_posh_git\profile.example.ps1")
}
Managing Pull Requests (Committers only)
As a committer, you frequently need to checkout pull requests in order to check them. Add the following set of functions to your profile to ease that process:
function REEF-Checkout-PullRequest{
param
(
[Parameter(Position=0, Mandatory=$true, HelpMessage='The pull request number')]
[int]$PR,
[Parameter(Mandatory=$false, HelpMessage='The branch to check the PR out into')]
[string]$B
)
$branch = if([string]::IsNullOrEmpty($B)) {"PR-$PR"} else {$B}
Invoke-Expression "git fetch https://github.com/apache/reef pull/$PR/head:$branch"
Invoke-Expression "git checkout $branch"
}
function REEF-Rebase-PullRequest{
Invoke-Expression 'git fetch apache; git rebase -i apache/master'
}
function REEF-Merge-PullRequest{
param
(
[Parameter(Position=0, Mandatory=$true, HelpMessage='The pull request number')]
[int]$PR
)
$branch = {"PR-$PR"}
Invoke-Expression "git checkout master"
Invoke-Expression "git merge $branch"
}
function REEF-Push-PullRequest{
Invoke-Expression 'git push apache master'
Invoke-Expression 'git push'
}
With this sequence of functions in your profile, the process of reviewing and merging pull request will look as follows:
REEF-Checkout-PullRequest N # ... test pull request ... REEF-Rebase-PullRequest # git commit --amend to fix commit message REEF-Merge-PullRequest N # last chance to change your mind and not merge! REEF-Push-PullRequest
Visual Studio
Install Visual Studio 2013, 2015, 2017. Then add the following to your PowerShell profile:
# Setup Visual Studio
function Setup-VisualStudio{
param
(
[Parameter(Mandatory=$true, HelpMessage='Version')]
$Version
)
$Script = "vcvarsall.bat&set"
if ($Version -eq "2013"){
$VisualStudioPath = 'c:\Program Files (x86)\Microsoft Visual Studio 12.0\VC'
}elseif ($Version -eq "2015"){
$VisualStudioPath = 'c:\Program Files (x86)\Microsoft Visual Studio 14.0\VC'
}elseif ($Version -eq "2017"){
$VisualStudioPath = 'C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Auxiliary\Build'
$Script = "vcvarsall.bat x64 & set"
}else{
throw "Please provide a Version of Visual Studio: 2013, 2015, 2017"
}
if(Test-Path $VisualStudioPath){
Push-Location $VisualStudioPath
cmd /c "$Script" |
foreach {
if ($_ -match "=") {
$v = $_.split("=")
Set-Item -Force -Path "ENV:\$($v[0])" -Value "$($v[1])"
}
}
Pop-Location
} else {
Write-Host "Visual Studio $Version not found at: $VisualStudioPath" -ForegroundColor Red
}
}
With this in your profile you can start Visual Studio with:
### 2013 Setup-VisualStudio 2013 ### 2015 Setup-VisualStudio 2015
Building (and cleaning) REEF
Assuming $REEFSourcePath points to the right folder, the following two functions can be used to build and reliably clean REEF:
function Build-REEF{
if($Env:REEFSourcePath){
Invoke-Expression 'msbuild $Env:REEFSourcePath\lang\cs\Org.Apache.REEF.sln /p:Configuration="Release" /p:Platform="x64" /m'
} else {
Write-Host "You must set REEFSourcePath to the root of your enclistment to use Build-REEF command" -ForegroundColor Red
}
}
function Clean-REEF{
if($Env:REEFSourcePath){
Invoke-Expression 'msbuild $Env:REEFSourcePath\lang\cs\Org.Apache.REEF.sln /p:Configuration="Release" /p:Platform="x64" /t:Clean /m'
Get-ChildItem -Path $Env:REEFSourcePath\lang\cs\ -Recurse -Filter obj | Remove-Item -Recurse
Get-ChildItem -Path $Env:REEFSourcePath\lang\cs\ -Recurse -Filter bin | Remove-Item -Recurse
Get-ChildItem -Path $Env:REEFSourcePath\lang\cs\ -Recurse -Filter target | Remove-Item -Recurse
Get-ChildItem -Path $Env:REEFSourcePath\lang\cs\ -Recurse -Filter REEF_LOCAL_RUNTIME | Remove-Item -Recurse
Get-ChildItem -Path $Env:REEFSourcePath\lang\cs\ -Recurse -Filter TestResults | Remove-Item -Recurse
Get-ChildItem -Path $Env:REEFSourcePath\lang\cs\ -Recurse -Filter packages | Remove-Item -Recurse
} else {
Write-Host "You must set REEFSourcePath to the root of your enlistment to use Clean-REEF command" -ForegroundColor Red
}
}
Verifying Release artifacts
We compute hashes for the releases using the GNU Core Utils. The functions below can be used to verify those hashes.
function Verify-Hash{
param
(
[Parameter(Mandatory=$true, HelpMessage='The .MD5 file.')]
[string]$File,
[Parameter(Mandatory=$true, HelpMessage='The Algorithm to use')]
[string]$Algorithm
)
foreach ($line in (Get-Content $File)) {
$fields = $line -split '\s+'
$hash = $fields[0].Trim().ToUpper()
$filename = $fields[1].Trim()
if($filename.StartsWith("*")){
$filename = $filename.Substring(1).Trim()
}
$computedHash = (Get-FileHash -Algorithm $Algorithm $filename).Hash.ToUpper()
if($hash.Equals($computedHash)){
Write-Host $filename, ": Passed"
}else{
Write-Host $filename, ": Not Passed"
Write-Host "Read from file: ", $hash
Write-Host "Computed: ", $computedHash
}
}
}
function Verify-MD5{
param
(
[Parameter(Position=0, Mandatory=$true, HelpMessage='The .MD5 file.')]
[string]$File
)
Verify-Hash -Algorithm MD5 -File $File
}
function Verify-SHA512{
param
(
[Parameter(Position=0, Mandatory=$true, HelpMessage='The .SHA512 file.')]
[string]$File
)
Verify-Hash -Algorithm SHA512 -File $File
}