job.md 11 KB
Job
submit
Build a federated learning job with two configuration files: job dsl and job conf, and submit it to the scheduler for execution
flow job submit [options]
Options
| parameter name | required | type | description |
|---|---|---|---|
| -d, --dsl-path | yes | string | path to job dsl |
| -c, --conf-path | yes | string | job conf's path |
Returns
| parameter name | type | description |
|---|---|---|
| retcode | int | return code |
| retmsg | string | return message |
| jobId | string | Job ID |
| data | dict | return data |
| data.dsl_path | string | The path to the actual running dsl configuration generated by the system based on the submitted dsl content |
| data.runtime_conf_on_party_path | string | The system-generated path to the actual running conf configuration for each party based on the submitted conf content |
| data.dsl_path | string | The system-generated path to the actual running conf configuration for each party based on the submitted conf content |
| data.board_url | string | fateboard view address |
| data.model_info | dict | Model identification information |
Example
{
"data": {
"board_url": "http://127.0.0.1:8080/index.html#/dashboard?job_id=202111061608424372620&role=guest&party_id=9999",
"code": 0,
"dsl_path": "$FATE_PROJECT_BASE/jobs/202111061608424372620/job_dsl.json",
"job_id": "202111061608424372620",
"logs_directory": "$FATE_PROJECT_BASE/logs/202111061608424372620",
"message": "success",
"model_info": {
"model_id": "arbiter-10000#guest-9999#host-10000#model",
"model_version": "202111061608424372620"
},
"pipeline_dsl_path": "$FATE_PROJECT_BASE/jobs/202111061608424372620/pipeline_dsl.json",
"runtime_conf_on_party_path": "$FATE_FATE_PROJECT_BASE/jobs/202111061608424372620/guest/9999/job_runtime_on_party_conf.json",
"runtime_conf_path":"$FATE_PROJECT_BASE/jobs/202111061608424372620/job_runtime_conf.json",
"train_runtime_conf_path": "$FATE_PROJECT_BASE/jobs/202111061608424372620/train_runtime_conf.json"
},
"jobId": "202111061608424372620",
"retcode": 0,
"retmsg": "success"
}
rerun
Rerun a job
flow job rerun [options]
Options
| parameter name | required | type | description | | :------------- | :------- | :--- | ----------- |------- | | -j, --job-id | yes | string | job id path | | --cpn, --component-name | no | string | Specifies which component to rerun from, unspecified components will not be executed if they have no upstream dependencies on the specified component; if not specified, the entire job will be rerun | | --force | no | bool | The job will be rerun even if it succeeds; if not specified, the job will be skipped if it succeeds |
Returns
| parameter name | type | description |
|---|---|---|
| retcode | int | return code |
| retmsg | string | return message |
| jobId | string | Job ID |
| data | dict | return data |
Example
flow job rerun -j 202111031100369723120
flow job rerun -j 202111031100369723120 -cpn hetero_lr_0
flow job rerun -j 202111031100369723120 -cpn hetero_lr_0 --force
parameter-update
Update the job parameters
flow job parameter-update [options]
Options
| parameter-name | required | type | description |
|---|---|---|---|
| -j, --job-id | yes | string | job id path |
| -c, --conf-path | yes | string | The contents of the job conf that needs to be updated, no need to fill in parameters that don't need to be updated |
Returns
| parameter name | type | description |
|---|---|---|
| retcode | int | return code |
| retmsg | string | return message |
| jobId | string | Job ID |
| data | dict | Returns the updated job conf |
Example
Assuming that the job is updated with some of the execution parameters of the hetero_lr_0 component, the configuration file is as follows.
{
"job_parameters": {
},
"component_parameters": {
"common": {
"hetero_lr_0": {
"alpha": 0.02,
"max_iter": 5
}
}
}
}
Execution of the following command takes effect.
flow job parameter-update -j 202111061957421943730 -c examples/other/update_parameters.json
Execute the following command to rerun.
flow job rerun -j 202111061957421943730 -cpn hetero_lr_0 --force
stop
Cancels or terminates the specified job
Options
| number | parameters | short format | long format | required parameters | parameter description |
|---|---|---|---|---|---|
| 1 | job_id | -j |
--job_id |
yes | Job ID |
Example
flow job stop -j $JOB_ID
query
Retrieve task information. Options
| number | parameters | short-format | long-format | required parameters | parameter description |
|---|---|---|---|---|---|
| 1 | job_id | -j |
--job_id |
no | Job ID |
| 2 | role | -r |
--role |
no | role |
| 3 | party_id | -p |
--party_id |
no | Party ID |
| 4 | status | -s |
--status |
No | Task status |
Example
flow job query -r guest -p 9999 -s complete
flow job query -j $JOB_ID
view
Retrieve the job data view. Options
| number | parameters | short-format | long-format | required parameters | parameter description |
|---|---|---|---|---|---|
| 1 | job_id | -j |
--job_id |
yes | Job ID |
| 2 | role | -r |
--role |
no | role |
| 3 | party_id | -p |
--party_id |
no | Party ID |
| 4 | status | -s |
--status |
No | Task status |
Example
flow job view -j $JOB_ID -s complete
config
Download the configuration file for the specified job to the specified directory.
Options
| number | parameters | short-format | long-format | required parameters | parameter description |
|---|---|---|---|---|---|
| 1 | job_id | -j |
--job_id |
yes | Job ID |
| 2 | role | -r |
--role |
yes | role |
| 3 | party_id | -p |
--party_id |
yes | Party ID |
| 4 | output_path | -o |
--output-path |
yes | output directory |
Example
flow job config -j $JOB_ID -r host -p 10000 --output-path . /examples/
log
Download the log file of the specified job to the specified directory. Options
| number | parameters | short-format | long-format | required parameters | parameter description |
|---|---|---|---|---|---|
| 1 | job_id | -j |
--job_id |
yes | Job ID |
| 2 | output_path | -o |
--output-path |
yes | output directory |
Example
flow job log -j JOB_ID --output-path . /examples/
list
Show the list of jobs. Options
| number | parameters | short-format | long-format | required parameters | parameter description |
|---|---|---|---|---|---|
| 1 | limit | -l |
-limit |
no | Returns the number limit (default: 10) |
Example
flow job list
flow job list -l 30
dsl
Predictive DSL file generator. Options
| number | parameters | short-format | long-format | required parameters | parameter description |
|---|---|---|---|---|---|
| 1 | cpn_list | -cpn-list |
No | List of user-specified component names | |
| 2 | cpn_path | -cpn-path |
No | User-specified path to a file with a list of component names | |
| 3 | train_dsl_path | -train-dsl-path |
yes | path to the training dsl file | |
| 4 | output_path | -o |
--output-path |
no | output directory path |
Example
flow job dsl --cpn-path fate_flow/examples/component_list.txt --train-dsl-path fate_flow/examples/test_hetero_lr_job_dsl.json
flow job dsl --cpn-path fate_flow/examples/component_list.txt --train-dsl-path fate_flow/examples/test_hetero_lr_job_dsl.json -o fate_flow /examples/
flow job dsl --cpn-list "dataio_0, hetero_feature_binning_0, hetero_feature_selection_0, evaluation_0" --train-dsl-path fate_flow/examples/ test_hetero_lr_job_dsl.json -o fate_flow/examples/
flow job dsl --cpn-list [dataio_0,hetero_feature_binning_0,hetero_feature_selection_0,evaluation_0] --train-dsl-path fate_flow/examples/ test_hetero_lr_job_dsl.json -o fate_flow/examples/