Q. I named my hadoop distribution file hadoop-2.6.0.tgz, but Myriad is trying to download a file named "binary.tgz"

A. Myriad renames the hadoop distro file to binary.tgz

 

Q. Why do I get this download error?

Download Error

Failed to fetch 'http://localhost:9192/api/artifacts/binary.tgz': Error downloading resource: Failure when receiving data from the peer
Failed to synchronize with slave (it's probably exited)

A. One possibility is that the binary.tgz file containing the hadoop distro has not been uploaded to hdfs per this configuration within myriad-config-default.yml:

Tar Files Configuration

servedConfigPath: dist/config.tgz
servedBinaryPath: dist/hadoop-2.6.0.tgz

So, in the case of this configuration example, simply upload the config.tgz file (that contains the etc/hadoop files) and the hadoop-2.6.0.tgz file (contains hadoop distribution being used) into the /dist directory of hdfs.

The other possibility is that the relative local directory path to the binary.tgz file–in this case, dist/hadoop-2.6.0.tgz–either does not exist or the directory from which you are launching the resource manager is not $YARN_HOME directory. For the former, create the directory and place the config.tgz and binary.tgz (again, in this case, hadoop-2.6.0.tgz) in that directory.  For the latter, cd to $YARN_HOME and then launch the Resource Manager via bin/yarn resourcemanager. 

 

Q. I have passwordless sudo for the frameworkUser and/or my frameworkUser is root, so why am I getting this error:

sudoless user error

sudo: sorry, you must have a tty to run sudo

A. This means the sudoers file default is set requiring a tty session as detailed here. To fix, do the following:

Fix tty error
sudo su root
visudo
#Comment or delete the following line:
Defaults    requiretty

 

Q, How do I examine the command Mesos uses to launch a YARN Resource Manager via a Myriad executor?

A. In analogy to all Mesos tasks, navigate to the Active Tasks panel from the Myriad Framework page, click on the Myriad task, and then select stdout. The stdout window contains the command that launches the Mesos task corresponding to the YARN Resource Manager and Myriad executor.

Q. Why do I get this max memory error:

max memory error

Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.yarn.exceptions.InvalidResourceRequestException): Invalid resource request, requested memory < 0, or requested memory > max configured, requestedMemory=1536, maxMemory=1024

    at org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerUtils.validateResourceRequest(SchedulerUtils.java:249)

    at org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerUtils.normalizeAndValidateRequest(SchedulerUtils.java:226)

    at org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.validateAndCreateResourceRequest(RMAppManager.java:376)

    at org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.createAndPopulateNewRMApp(RMAppManager.java:320)

    at org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.submitApplication(RMAppManager.java:273)

    at org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitApplication(ClientRMService.java:574)

    at org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.submitApplication(ApplicationClientProtocolPBServiceImpl.java:213)

    at org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:403)

    at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)

A. This means that all of the Myriad tasks--NodeManagers–are allocated less RAM than the either the default mapreduce.map.memory.mb (in this case, 1536) or the value of this parameter specified in the mapred-site.xml.

  • No labels