This is usually caused by an _ in the host name. An example: PC_DEV2 would be a valid windows host name, but if you try to start ServiceMix on such a machine, you would get:
java.io.IOException: Transport Connector could not be registered in JMX: illegal character in hostname at index 8: tcp://PC_DEV2:61616 ... Caused by: java.net.URISyntaxException: Illegal character in hostname at position x
The solution would be to change the host name of the machine that is running ServiceMix (just replace the _ by - which is a valid character)