Message Schemas¶
Before you release your application, you should create a subclass of
fedora_messaging.message.Message, define a schema, define a default
severity, and implement some methods.
Schema¶
Defining a message schema is important for several reasons.
First and foremost, if will help you (the developer) ensure you don’t accidentally change your message’s format. When messages are being generated from, say, a database object, it’s easy to make a schema change to the database and unintentionally alter your message, which breaks consumers of your message. Without a schema, you might not catch this until you deploy your application and consumers start crashing. With a schema, you’ll get an error as you develop!
Secondly, it allows you to change your message format in a controlled fashion by versioning your schema. You can then choose to implement methods one way or another based on the version of the schema used by a message. For details on how to deprecate and upgrade message schemas, see Upgrade and deprecation.
Message schema are defined using JSON Schema. The complete API can be found in the Message Schemas API documentation.
Header Schema¶
The default header schema declares that the header field must be a JSON object with several expected keys. You can leave the schema as-is when you define your own message, or you can refine it. The base schema will always be enforced in addition to your custom schema.
Body Schema¶
The default body schema simply declares that the header field must be a JSON object.
Example Schema¶
# Copyright (C) 2018 Red Hat, Inc.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License along
# with this program; if not, write to the Free Software Foundation, Inc.,
# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
"""This is an example of a message schema."""
from email.utils import parseaddr
from fedora_messaging import message
class BaseMessage(message.Message):
"""
You should create a super class that each schema version inherits from.
This lets consumers perform ``isinstance(msg, BaseMessage)`` if they are
receiving multiple message types and allows the publisher to change the
schema as long as they preserve the Python API.
"""
def __str__(self):
"""Return a complete human-readable representation of the message."""
return "Subject: {subj}\n{body}\n".format(
subj=self.subject, body=self.email_body
)
@property
def summary(self):
"""Return a summary of the message.
By convention, in Fedora all schemas should provide this property.
"""
return self.subject
@property
def subject(self):
"""The email's subject."""
return 'Message did not implement "subject" property'
@property
def email_body(self):
"""The email message body."""
return 'Message did not implement "email_body" property'
@property
def url(self):
"""An URL to the email in HyperKitty
By convention, in Fedora all schemas should provide this property.
Returns:
str or None: A relevant URL.
"""
base_url = "https://lists.fedoraproject.org/archives"
archived_at = self._get_archived_at()
if archived_at and archived_at.startswith("<"):
archived_at = archived_at[1:]
if archived_at and archived_at.endswith(">"):
archived_at = archived_at[:-1]
if archived_at and archived_at.startswith("http"):
return archived_at
elif archived_at:
return base_url + archived_at
else:
return None
@property
def app_name(self):
"""The name of the application that generated the message.
By convention, in Fedora all schemas should provide this property.
"""
return "Mailman"
@property
def app_icon(self):
"""A URL to the icon of the application that generated the message.
By convention, in Fedora all schemas should provide this property.
"""
return "https://apps.fedoraproject.org/img/icons/hyperkitty.png"
@property
def usernames(self):
"""List of users affected by the action that generated this message."""
return []
@property
def packages(self):
"""List of packages affected by the action that generated this message."""
return []
def _get_username_from_from_header(self, from_header):
"""Converts a From email header to a username."""
# Extract the username
addr = parseaddr(from_header)[1]
return addr.split("@")[0]
class MessageV1(BaseMessage):
"""
A sub-class of a Fedora message that defines a message schema for messages
published by Mailman when it receives mail to send out.
"""
body_schema = {
"id": "http://fedoraproject.org/message-schema/mailman#",
"$schema": "http://json-schema.org/draft-04/schema#",
"description": "Schema for message sent to mailman",
"type": "object",
"properties": {
"mlist": {
"type": "object",
"properties": {
"list_name": {
"type": "string",
"description": "The name of the mailing list",
}
},
},
"msg": {
"description": "An object representing the email",
"type": "object",
"properties": {
"delivered-to": {"type": "string"},
"from": {"type": "string"},
"cc": {"type": "string"},
"to": {"type": "string"},
"x-mailman-rule-hits": {"type": "string"},
"x-mailman-rule-misses": {"type": "string"},
"x-message-id-hash": {"type": "string"},
"references": {"type": "string"},
"in-reply-to": {"type": "string"},
"message-id": {"type": "string"},
"archived-at": {"type": "string"},
"subject": {"type": "string"},
"body": {"type": "string"},
},
"required": ["from", "to", "subject", "body"],
},
},
"required": ["mlist", "msg"],
}
@property
def subject(self):
"""The email's subject."""
return self.body["msg"]["subject"]
@property
def email_body(self):
"""The email message body."""
return self.body["msg"]["body"]
@property
def agent_name(self):
"""The username of the user who caused the action."""
return self._get_username_from_from_header(self.body["msg"]["from"])
def _get_archived_at(self):
return self.body["msg"]["archived-at"]
class MessageV2(BaseMessage):
"""
This is a revision from the MessageV1 schema which flattens the message
structure into a single object, but is backwards compatible for any users
that make use of the properties (``subject`` and ``body``).
"""
body_schema = {
"id": "http://fedoraproject.org/message-schema/mailman#",
"$schema": "http://json-schema.org/draft-04/schema#",
"description": "Schema for message sent to mailman",
"type": "object",
"required": ["mailing_list", "from", "to", "subject", "body"],
"properties": {
"mailing_list": {
"type": "string",
"description": "The name of the mailing list",
},
"delivered-to": {"type": "string"},
"from": {"type": "string"},
"cc": {"type": "string"},
"to": {"type": "string"},
"x-mailman-rule-hits": {"type": "string"},
"x-mailman-rule-misses": {"type": "string"},
"x-message-id-hash": {"type": "string"},
"references": {"type": "string"},
"in-reply-to": {"type": "string"},
"message-id": {"type": "string"},
"archived-at": {"type": "string"},
"subject": {"type": "string"},
"body": {"type": "string"},
},
}
@property
def subject(self):
"""The email's subject."""
return self.body["subject"]
@property
def email_body(self):
"""The email message body."""
return self.body["body"]
@property
def agent_name(self):
"""The username of the user who caused the action."""
return self._get_username_from_from_header(self.body["from"])
def _get_archived_at(self):
return self.body["archived-at"]
Note that message schema can be composed of other message schema, and validation of fields can be much more detailed than just a simple type check. Consult the JSON Schema documentation for complete details.
Message Conventions¶
Schema are Immutable¶
Message schema should be treated as immutable. Once defined, they should not be altered. Instead, define a new schema class, mark the old one as deprecated, and remove it after an appropriate transition period.
Provide Accessors¶
The JSON schema ensures the message sent “on the wire” conforms to a particular format. Messages should provide Python properties to access the deserialized JSON object. This Python API should maintain backwards compatibility between schema. This shields consumers from changes in schema.
Useful Accessors¶
All available accessors are described in the Message Schemas API documentation ; here is a list of those we recommend implementing to allow users to get notifications for your messages:
__str__(): A human-readable representation of this message. This can be a multi-line string that forms the body of an email notification.summary: A short, single-line, human-readable summary of the message, much like the subject line of an email.agent_name: The username of the user who caused the action.app_name: The name of the application that generated the message. This can be implemented as a class attribute or as a property.app_icon: A URL to the icon of the application that generated the message. This can be implemented as a class attribute or as a property.packages: A list of RPM packages affected by the action that generated this message, if any.flatpaks: A list of flatpaks affected by the action that generated this message, if any.modules: A list of modules affected by the action that generated this message, if any.containers: A list of containers affected by the action that generated this message, if any.usernames: A list of usernames affected by the action that generated this message. This may contain theagent_name.groups: A list of group names affected by the action that generated this message.url: A URL to the action that caused this message to be emitted, if any.severity: An integer that indicates the severity of the message. This is used to determine what messages to notify end users about and should beDEBUG,INFO,WARNING, orERROR. The default isINFO, and can be set as a class attribute or on an instance-by-instance basis.
Packaging¶
Finally, you must distribute your schema to clients. It is recommended that you
maintain your message schema in your application’s git repository in a separate
Python package. The package name should be <your-app-name>-messages.
A complete sample schema package can be found in the fedora-messaging repository. This includes unit tests, the schema classes, and a setup.py. You can adapt this boilerplate with the following steps:
Change the package name from
mailman_messagesto<your-app-name>_messagesinsetup.py.Rename the
mailman_messagesdirectory to<your-app-name>_messages.Add your schema classes to
messages.pyand tests totests/test_messages.py.Update the
READMEfile.Build the distribution with
python setup.py sdist bdist_wheel.Upload the sdist and wheel to PyPI with
twine.Submit an RPM package for it to Fedora and EPEL.
If you prefer CookieCutter, there is a template repository that you can use with the command:
cookiecutter gh:fedora-infra/cookiecutter-message-schemas
It will ask you for the application name and some other variables, and will create the package structure for you.
Upgrade and deprecation¶
Message schema classes should not be modified in a backwards-incompatible fashion. To facilitate the
evolution of schemas, we recommend including the schema version in the topic itself, such as
myservice.myevent.v1.
When a backwards-incompatible change is required, create a new class with the topic ending in
.v2, set the Message.deprecated attribute to True on the old class, and send both
versions for a reasonable period of time. Note that you need to add the new class to the schema
package’s entry points as well.
We leave the duration to the developer’s appreciation, since it depends on how many different consumers they expect to have, whether they are only inside the Fedora infrastructure or outside too, etc. This duration can range from weeks to months, possibly a year. At the time of this writing, Fedora’s message bus is very far from being overwhelmed by messages, so you don’t need to worry about that.
Proceeding this way ensures that consumers subscribing to .v1 will not break when .v2
arrives, and can choose to subscribe to the .v2 topic when they are ready to handle the new
format. They will get a warning in their logs when they receive deprecated messages, prompting them
to upgrade.
When you add the new version, please upgrade the major version number of your schema package, and communicate clearly that the old version is deprecated, including for how long you have decided to send both versions.