PigStorage also load/store complex data type: tuple, bag, map. Here are formats:
- Tuple: enclosed by (), items separated by ",". Eg: (item1,item2,item3). Empty tuple is valid. Eg: ()
- Bag: enclosed by {}, tuples separated by ",". Eg: {code}{(tuple1),(tuple2),(tuple3)}{code}. Empty bag is valid. Eg: {}
- Map: enclosed by [], items separated by ",", key and value seperated by "#". Eg: [key1#value1,key2#value2]. Empty map is valid. Eg: []
If load statement specify a schema, Pig will convert the complex type according to schema. If conversion fail, the affected item will be null.
Here are more examples:
a = load '1.txt' as (a0:{t:(m:map[int],d:double)});
{([foo#1,bar#2],34.0),([white#3,yellow#4],45.0)} : valid
{([foo#badint],baddouble)} : conversion fail for badint/baddouble, get {([foo#],)}
{} : valid, empty bag