exampleservice¶
The easiest way to explain how to use fastapi-serviceutils is to demonstrate usage inside an exampleservice. Here we will explain the parts of the service and which functions and classes when to use.
Creating new service¶
To create a new service we use the tool create_service
which is available
after installing fastapi-serviceutils.
create_service -n exampleservice \
-p 50001 \
-a "Dummy User" \
-e dummy.user@something.info \
-ep example \
-o /tmp
This creates the service exampleservice inside the folder /tmp/exampleservice. As author with email we define Dummy User and dummy.user@something.info. The initial endpoint we want to create is example. The service should listen to port 50001.
If we change into the created directory we will have the following folder-structure:
exampleservice
├── app
│ ├── config.yml
│ ├── endpoints
│ │ ├── __init__.py
│ │ └── v1
│ │ ├── errors.py
│ │ ├── example.py
│ │ ├── __init__.py
│ │ └── models.py
│ ├── __init__.py
│ └── main.py
├── .codespell-ignore-words.txt
├── docker-compose.yml
├── Dockerfile
├── docs
│ └── ...
├── .gitignore
├── Makefile
├── .pre-commit-config.yaml
├── pyproject.toml
├── .python-version
├── README.md
├── setup.cfg
├── tests
│ └── __init__.py
└── .tmuxp.yml
The files docker-compose.yml
and Dockerfile
are required for
deployment of the service as docker-container.
.tmuxp.yml
is used for development of the service if you prefer to develop
inside tmux in combination with for example vim or emacs.
The .python-version
defines which python-version this service uses and is
used by poetry / dephell workflow inside virtual-environments.
The pyproject.toml
is used for dependency-management and package-creation.
setup.cfg
contains configurations for tools used during development like
yapf, flake8, pytest, etc.
The .pre-commit-config.yaml
allows the usage of pre-commit and is also
used in the make command make check
.
It enables running of multiple linters, checkers, etc. to ensure a fixed
codestyle.
The Makefile
contains helper command like initializing the project,
updating the virtual-environment, running tests, etc.
Because codespell is used inside the configuration of pre-commit, the file
.codespell-ignore-words.txt
is used to be able to define words to be
ignored during check with codespell.
Initialising project¶
To initialise the project after creation we run:
make init
This creates the virtual-environment and installs the dependencies as defined
in the pyproject.toml
.
It also initialises the project as a git-folder and creates the initial
commit.
We now activate the poetry-shell to enable the environment:
poetry shell
Attention
Please ensure to always enable the poetry-shell before development using:
poetry shell
The Makefile assumes the environment is activated on usage.
Folder-structure¶
Following shows code-relevant files for an exampleservice as created using the create_service-tool of fastapi-serviceutils.
exampleservice
├── app
│ ├── config.yml
│ ├── endpoints
│ │ ├── __init__.py
│ │ └── v1
│ │ ├── errors.py
│ │ ├── example.py
│ │ ├── __init__.py
│ │ └── models.py
│ ├── __init__.py
│ └── main.py
├── pyproject.toml
└── tests
├── __init__.py
└── service_test.py
pyproject.toml¶
The dependencies and definitions like the package-name, version, etc. are
defined inside the pyproject.toml
.
This file is used by Poetry and Dephell.
Following the pyproject.toml
for our exampleservice:
[tool.poetry]
name = "exampleservice"
version = "0.1.0"
description = "Exampleservice to demonstrate usage of fastapi-serviceutils."
authors = ["Dummy User <dummy.user@something.info>"]
readme = "README.md"
include = ["README.md", "app/config.yml"]
[tool.poetry.dependencies]
python = ">=3.7,<4"
fastapi-serviceutils = ">=2"
[tool.poetry.dev-dependencies]
autoflake = ">=1.3"
coverage-badge = ">=1"
flake8 = ">=3.7"
ipython = ">=7.8"
isort = ">=4.3"
jedi = ">=0.14"
neovim = ">=0.3.1"
pre-commit = ">=1.18.3"
pudb = ">=2019.1"
pygments = ">=2.4"
pytest = ">=5"
pytest-asyncio = ">=0.10"
pytest-cov = ">=2"
pytest-xdist = ">=1.30"
sphinx = ">=2"
sphinx-autodoc-typehints = ">=1.6"
sphinx-rtd-theme = ">=0.4.3"
yapf = ">=0.27"
[tool.poetry.extras]
devs = [
"autoflake", "coverage", "coverage-badge", "flake8", "ipython", "isort",
"jedi", "neovim", "pre-commit", "pudb", "pygments", "pytest",
"pytest-asyncio", "pytest-cov", "pytest-xdist", "sphinx",
"sphinx-autodoc-typehints", "sphinx-rtd-theme", "yapf"
]
[tool.dephell.devs]
from = {format = "poetry", path = "pyproject.toml"}
envs = ["main", "devs"]
[tool.dephell.main]
from = {format = "poetry", path = "pyproject.toml"}
to = {format = "setuppy", path = "setup.py"}
envs = ["main"]
versioning = "semver"
[tool.dephell.lock]
from = {format = "poetry", path = "pyproject.toml"}
to = {format = "poetrylock", path = "poetry.lock"}
[tool.poetry.scripts]
exampleservice = "app.main:main"
[build-system]
requires = ["poetry>=0.12"]
build-backend = "poetry.masonry.api"
app/config.yml¶
The service is configured using a config-file (config.yml
).
It is possible to overwrite these setting using environment-variables.
An example for the config.yml
of the exampleservice is shown below:
service:
name: 'exampleservice'
mode: 'devl'
port: 50001
description: 'Example tasks'
apidoc_dir: 'docs/_build'
readme: 'README.md'
allowed_hosts:
- '*'
use_default_endpoints:
- alive
- config
external_resources:
services: null
databases: null
other: null
logger:
path: './log/EXAMPLESERVICE'
filename: 'service_{mode}.log'
level: 'debug'
rotation: '1 days'
retention: '1 months'
format: "<green>{time:YYYY-MM-DD HH:mm:ss.SSS}</green> | <level>{level: <8}</level> | <cyan>{name}</cyan>:<cyan>{function}</cyan>:<cyan>{line}</cyan> [id: {extra[request_id]}] - <level>{message}</level>"
available_environment_variables:
env_vars:
- SERVICE__MODE
- SERVICE__PORT
- LOGGER__LEVEL
- LOGGER__PATH
- LOGGER__FILENAME
- LOGGER__ROTATION
- LOGGER__RETENTION
- LOGGER__FORMAT
external_resources_env_vars:
- EXTERNAL_RESOURCES__API__URL
- EXTERNAL_RESOURCES__API__SCHEMA
rules_env_vars: []
The config contains four main sections:
service
external_resources
logger
available_environment_variables
config: [service]¶
Inside this section we define the name of the service name
.
This name is used for the swagger-documentation and extraction of the
environment-variables.
The mode
define the runtime-mode of the service.
This mode can be overwritten with the environment-variable
EXAMPLESERVICE__SERVICE__MODE
(where 'EXAMPLESERVICE'
is the name of
the service, meaning if you have a service named SOMETHING
the
environment-variable would be SOMETHING__SERVICE__MODE
).
The port
configure the port the service will listen to.
This can also be overwritten using the environment variable
EXAMPLESERVICE__SERVICE__PORT
.
The description
is used for the swagger-documentation.
To define the folder where the to find the apidoc to serve by route
/api/apidoc/index.html
the keyword apidoc_dir
is used.
readme
defines where to get the readme from to be used as main description
for the swagger-documentation at /docs
/ /redoc
.
To controll if only specific hosts are allowed to controll the service we use
allowed_hosts
.
Per default a service would allow all hosts ('*'
) but this can be
customized here in the config.
To define which default endpoints should be included in our service we use
use_default_endpoints
.
Currently we support the default endpoints /api/alive
(inside config:
'alive'
) and /api/config
(inside config: 'alive'
).
config: [external_resources]¶
Inside this section external dependencies (resources) are defines. A service can depend on other services, databases, remote-connections or files / folders.
Dependencies to other services should be defined inside services
.
Database connections inside databases
(currently only postgres is
supported).
If any other dependency exist define it in other
.
Defined services can be accessed in the code using
app.config.external_resources.services
or
ENDPOINT.config.external_resources.services
depending if you are in a main
part of the app or inside an endpoint.
Databases are automatically included into the startup
and shutdown
handlers.
You can access the database connection using app.databases['DATABASE_NAME']
or ENDPOINT.databases['DATABASE_NAME']
depending if you are in a main part
of the app or inside an endpoint.
config: [logger]¶
All settings inside this section are default Loguru settings to configure the
logger.
You can control where to log (path
) and how the logfile should be named
(filename
).
Also which minimum level to log (level
).
To control when to rotate the logfile use rotation
.
retention
defines when to delete old logfiles.
The format
defines the format to be used for log-messages.
config: [available_environment_variables]¶
The environment-variables are seperated into three types:
env_vars
external_resources_env_vars
rules_env_vars
Here you can control which environment-variables to use if they are set.
The environment-variables are named like the following:
<SERVICENAME>__<MAJOR_SECTION>__<PARAMETER_NAME>
.
The servicename would be 'EXAMPLESERVICE'
in our example.
The major-section is one of:
'SERVICE'
'LOGGER'
'EXTERNAL_RESOURCES'
env_vars
control the sections service
and logger
.
external_resources_env_vars
control the configurations inside the section
external_resources
.
The rules_env_vars
should overwrite settings of a ruleset of the service.
Such a ruleset defines constants and other rules for the logic of the
endpoints.
For example a default time-range for your pandas dataframes, etc.
Currently this is not implemented, so you would have to use these definitions
yourself to overwrite your ruleset-definitions.
app/__init__.py¶
Inside the __init__.py
file of the app we only define the version of our
service.
Note
We use semantic-versioning style for services based on fastapi-serviceutils.
This means we have the following version-number:
<MAJOR>.<MINOR>.<PATCH>
.
For details about semantic-versioning see Semver.
If we bump the version using either dephell bump {major, minor, fix}
or
poetry version {major, minor, patch}
, both the version defined here, and
the version defined inside the pyproject.toml
will be increased.
__version__ = '0.1.0'
app/main.py¶
Inside this file we glue all parts of our service together.
Here the app
is created which is used either in development inside the
function main
or in production using uvicorn
from command line (or
docker-container).
from pathlib import Path
from typing import NoReturn
from app import __version__
from app.endpoints import ENDPOINTS
from fastapi_serviceutils import make_app
app = make_app(
config_path=Path(__file__).with_name('config.yml'),
version=__version__,
endpoints=ENDPOINTS,
enable_middlewares=['trusted_hosts',
'log_exception'],
additional_middlewares=[]
)
def main() -> NoReturn:
import uvicorn
uvicorn.run(app, host='0.0.0.0', port=app.config.service.port)
if __name__ == '__main__':
main()
We define where to collect the config-file of the service from, the version of the service and which endpoints and middlewares to use.
app/endpoints/v1/example.py¶
The following shows the example-endpoint we created:
from app.endpoints.v1.models import Example as Output
from app.endpoints.v1.models import GetExample as Input
from fastapi import APIRouter
from fastapi import Body
from starlette.requests import Request
from fastapi_serviceutils.app import create_id_logger
from fastapi_serviceutils.app import Endpoint
ENDPOINT = Endpoint(router=APIRouter(), route='/example', version='v1')
SUMMARY = 'Example request.'
EXAMPLE = Body(..., example={'msg': 'some message.'})
@ENDPOINT.router.post('/', response_model=Output, summary=SUMMARY)
async def example(request: Request, params: Input = EXAMPLE) -> Output:
_, log = create_id_logger(request=request, endpoint=ENDPOINT)
log.debug(f'received request for {request.url} with params {params}.')
return Output(msg=params.msg)
The ENDPOINT
includes the router
, route
and the version
of our
endpoint.
Inside the endpoint-function we create a new bound logger with the request-id of the request to allow useful traceback.
Note
Defining endpoints like this allows our worklow with endpoint-versioning
and usage of fastapi_serviceutils.endpoints.set_version_endpoints()
inside app/endpoints/v1/__init__.py
and app/endpoints/__init__.py
.
app/endpoints/v1/models.py¶
The models.py contains models for the endpoints in version 1 of our exampleservice.
For each endpoint we create the model for the input (request) and the model for the output (response).
The models are of type pydantic.BaseModel
from pydantic import BaseModel
class GetExample(BaseModel):
msg: str
class Example(BaseModel):
msg: str
__all__ = ['Example', 'GetExample']
More complex models could look like the following:
"""
In special cases also an ``alias_generator`` has to be defined.
An example for such a special case is the attribute ``schema`` of
:class:`SpecialParams`. The schema is already an attribute of a BaseModel,
so it can't be used and an alias is required.
To be able to add post-parse-methods the pydantic ``dataclass`` can be
used.
An example for this can be seen in :class:`Complex`.
"""
from pydantic import BaseModel
from pydantic import Schema
from pydantic.dataclasses import dataclass
@dataclass
class Complex:
"""Represent example model with attribute-change of model after init."""
accuracy: str
def __post_init_post_parse__(self) -> NoReturn:
"""Overwrite self.accuracy with a mapping as defined below."""
accuracy_mapping = {
'something': 's',
'match': 'm',
}
self.accuracy = accuracy_mapping[self.accuracy]
def _alias_for_special_model_attribute(alias: str) -> str:
"""Use as ``alias_generator`` for models with special attribute-names."""
return alias if not alias.endswith('_') else alias[:-1]
class SpecialParams(BaseModel):
"""Represent example model with special attribute name requiring alias."""
msg: str
schema_: str = Schema(None, alias='schema')
class Config:
"""Required for special attribute ``schema``."""
alias_generator = _alias_for_special_model_attribute
app/endpoints/v1/__init__.py¶
Inside this file we include our example-endpoint to the version 1 endpoints.
Note
If additional endpoints are available, these should be added here, too.
The created ENDPOINTS
is used inside app/endpoints/__init__.py
later.
Note
If we would increase our version to version 2 and we want to change the
endpoint example
we would add an additional folder inside
app/endpoints
named v2
and place the new version files there.
from app.endpoints.v1 import example
from fastapi_serviceutils.app.endpoints import set_version_endpoints
ENDPOINTS = set_version_endpoints(
endpoints=[example],
version='v1',
prefix_template='/api/{version}{route}'
)
__all__ = ['ENDPOINTS']
app/endpoints/__init__.py¶
In this file we import all endpoint-versions like in this example
from app.endpoints.v1 import ENDPOINTS as v1
.
Note
If we would have an additional version 2 we would also add from
app.endpoints.v2 import ENDPOINTS as v2
.
Then we use fastapi_serviceutils.endpoints.set_version_endpoints()
with
the latest version endpoints to create LATEST
.
Note
If we would have version 2, too we would replace parameter endpoints
with v2
.
The ENDPOINTS
is a list of all available versions.
These ENDPOINTS
are used inside app/main.py
to include them to the
service.
from app.endpoints.v1 import ENDPOINTS as v1
from fastapi_serviceutils.app.endpoints import set_version_endpoints
LATEST = set_version_endpoints(
endpoints=v1,
version='latest',
prefix_template='{route}'
)
ENDPOINTS = LATEST + v1
__all__ = ['ENDPOINTS']
tests¶
The tests for the exampleservice are using Pytest.
We also used the testutils of fastapi-serviceutils
.
An example for simple endpoint tests of our exampleservice:
import pytest
from app.main import app
from fastapi_serviceutils.app.service_config import Config
from fastapi_serviceutils.utils.tests.endpoints import json_endpoint
def test_endpoint_example():
json_endpoint(
application=app,
endpoint='/api/v1/example/',
payload={'msg': 'test'},
expected={'msg': 'test'}
)
@pytest.mark.parametrize(
'endpoint, status_code',
[
('/api/v1/example',
307),
('/api/',
404),
('/api/v1/',
404),
('/api/v1/example/',
200),
]
)
def test_endpoint_invalid(endpoint, status_code):
json_endpoint(
application=app,
endpoint=endpoint,
status_code=status_code,
payload={'msg': 'test'}
)