Here I went to a meetup that talked about workflow management tools namely Azkaban and Oozie.
So imagine you have a long process of jobs in different programming languages, libraries etc. and you are handling each time e.g. manually run them or check their exit status, or write a shell script with ifs/elses to proceed to the rest of the jobs. That is UGLY!
So Azkaban will take care o this for you in a nice GUI interface and tell you which ones succeeded or failed, and email you if they failed or succeeded. Now I like that!
Apparently it is not able to take care of creating directories or etc based on the workflow job so for those stuff you still need to have your own bash script.
No comments:
Post a Comment