This page describes a set of conventions for implementing APIs in Airflow. It is based on the following documents:

Use plural nouns but no verbs

ResourceGET
read		POST
create		PUT
update		DELETE
/dagsReturns a list of dagsCreate a new dagBulk update of dagsDelete all dags
/dags/711Returns a specific dagMethod not allowed (405)Updates a specific dagDeletes a specific dag

GET method and query parameters should not alter the state

Use PUT, POST and DELETE methods  instead of the GET method to alter the state.
Do not use GET for state changes:

Public facing endpoints should return JSON

 

GET /dags

{

    “dag”: {

        “id”: “my_dag”,

        “owner”: “airflow”,

        “start_date”: “2012-04-23T18:25:43.511Z”,

        …

    }

}

Service endpoints should return (tbd)

  • Avro or protobuf (to be decided)
  •  

 Use HTTP headers for serialization formats

Both, client and server, need to know which format is used for the communication. The format has to be specified in the HTTP-Header.

Content-Type defines the request format.
Accept defines a list of acceptable response formats.

Version your endpoint

Endpoint versioning is mandatory. Use a simple ordinal number and avoid dot notation such as 2.5. If you think a particular functionality is still experimental, version it as such. 

/api/v1/dags

/api/experimental/features

We might want to maintain an unversioned link that points to the latest version of an endpoint (t.b.d.)

 

Handle Errors with HTTP status codes

 

It is hard to work with an API that ignores error handling. Pure returning of a HTTP 500 with a stacktrace is not very helpful.

 

Use HTTP status codes

 

The HTTP standard provides over 70 status codes to describe the return values. We don’t need them all, but  there should be used at least a mount of 10.

 

200 – OK – Eyerything is working
201 – OK – New resource has been created
204 – OK – The resource was successfully deleted

 

304 – Not Modified – The client can use cached data

 

400 – Bad Request – The request was invalid or cannot be served. The exact error should be explained in the error payload. E.g. „The JSON is not valid“
401 – Unauthorized – The request requires an user authentication
403 – Forbidden – The server understood the request, but is refusing it or the access is not allowed.
404 – Not found – There is no resource behind the URI.
422 – Unprocessable Entity – Should be used if the server cannot process the enitity, e.g. if an image cannot be formatted or mandatory fields are missing in the payload.

 

500 – Internal Server Error – API developers should avoid this error. If an error occurs in the global catch blog, the stracktrace should be logged and not returned as response.

 

Use error payloads

 

All exceptions should be mapped in an error payload.

Provide authentication for your endpoint

work in progress

Public facing endpoints

  • Standard (web) authentication methods
  • Kerberos

Internal endpoints

  • Kerberos
  • None
  • Simple
  • OAUTH2











  • No labels