0

The class TaskStatus class has a method getProgress() but I am not sure on what this method actually returns.

Is it the time that was spent or is it input data that was processed?

I want to know how much data was processed until now in the source code. What should I do? Should I use getProgress()?

David Guyon
  • 2,759
  • 1
  • 28
  • 40
mndn
  • 73
  • 8

1 Answers1

1

This is the JavaDoc comment from here

Get the current progress.

Returns: a number between 0.0 and 1.0 (inclusive) indicating the attempt's progress.

Attempt being the mapreduce job here.

Community
  • 1
  • 1
philantrovert
  • 9,904
  • 3
  • 37
  • 61
  • how can I get the input data size of each map or reduce task ? – mndn May 23 '17 at 11:32
  • @mndn I don't know of any function that calculates the exact size, but, for mapper the size of input data will always be less than the split size which you can calculate using `FileInputFormat.computeSplitSize()` – philantrovert May 23 '17 at 12:29
  • TaskStatus class has a method getPhase(). what does it return?(are they only map, shuffle, sort, reduce?) – mndn May 29 '17 at 05:26
  • @mndn It can return anything from `STARTING, MAP, SHUFFLE, SORT, REDUCE, CLEANUP` depending on he current `Phase` which is an enum. – philantrovert May 29 '17 at 06:51
  • I need to call FileInputFormat.computeSplitSize() in defualtSpeculator.java,it need k,v (FileInputFormat).How I can do that ? – mndn May 30 '17 at 06:45
  • @mndn AFAIK, this `` denotes Java's `Generics`. You don't need to pass those `K` and `V`. Just `import org.apache.hadoop.mapred.FileInputFormat` and you should be able to use `computeSplitSize` – philantrovert May 30 '17 at 06:54
  • you mean I should write FileInputFormat.computeSplitSize() ? – mndn May 30 '17 at 07:02
  • Nope, I don't think so. Once you have imported the class, you can call the function just by it's name. It should be `computeSplitSize` only – philantrovert May 30 '17 at 07:03
  • I could not.I add a photo of what I did to the top question. – mndn May 30 '17 at 07:09
  • @mndn I suggest you ask a different question for that. – philantrovert May 30 '17 at 07:09
  • I asked, my latest question .would you please answer there.thanks. – mndn May 30 '17 at 07:11
  • https://stackoverflow.com/questions/44255386/how-can-i-get-the-input-data-size-of-each-map-or-reduce-task – mndn May 30 '17 at 07:13