Say Hi to Prefect Discourse — a New Forum for Data Engineers

Say Hi to Prefect Discourse — a New Forum for Data Engineers

February 16, 2022
PrefectData EngineeringOpen-SourceCommunity

Say Hi to Prefect Discourse — a New Forum for Data Engineers


We're excited to announce that we have launched Prefect Discourse! Discourse is a forum for online communities that provides an unmatched level of flexibility and customization. Going forward, we will use Discourse as a knowledge base that we can grow together as a community. It's a safe place to ask any questions about Prefect and workflow orchestration and share your knowledge with others. You can expect a wealth of tutorials, frequently asked questions, code examples, blog posts, announcements, troubleshooting tips, and in-depth technical discussions. Sign up today on

What do I gain from Discourse as a Prefect user?

Discourse does many things Slack can’t do:

  • It's easier to find relevant information through the docs, tags, categories, and a global search bar.
  • Easier embedding of screenshots and code blocks from GitHub for troubleshooting.
  • Syntax highlighting when you share code blocks with us, which makes discussing code that much easier,
  • Easier quoting and cross-linking of topics and external resources.
  • Much better categorization of content and support questions bringing more structure for everyone involved.
  • You can get notified about the latest posts related to specific Prefect features and integrations.
  • Google-searchable tutorials, frequently asked questions, and code examples, allowing you (in the near future) to google any question about Prefect and find a relevant Discourse topic with a potential solution to your issue.

You can think of Discourse as a knowledge base we can grow together as a community. To find out more about Discourse, check out this Getting Started with Prefect Discourse guide and for more background about this decision, read on.

Why are we introducing yet another platform?

Prefect has a remarkable, exceptionally active, and quickly growing community of engineers and data professionals. With this incredible growth, we've realized that relying solely on Slack has some downsides.

1) The lost knowledge

The free tier of Slack allows keeping the history of 10,000 most recent messages. While this might seem like a lot at first, with such an active and quickly growing community like ours, new 10,000 posts are written within two months. This way, all the valuable conversations and knowledge shared by our users get lost in no time.

We are lucky to have Marvin, brain the size of a planet, capable of automatically archiving Slack conversations as GitHub issues.


But archiving Slack discussions into GitHub issues doesn't entirely solve the problem. Such automatically archived discussions are less readable (lost images, code blocks, user icons), and they are not tagged in any way, leaving the problem of discoverability almost entirely unaddressed.

2) No categorization

When using Slack, it's usually easier (and faster!) to repeat the same question than trying to find if a similar question has already been asked. There are several reasons for it:

  1. Slack is optimized for writing. It's a chat platform, so it's natural that it makes creating new messages the number one priority.
  2. The search functionality is quite limited. There is a full-text search, but it's often challenging to find, for example, all questions about a specific feature or problem.
  3. Slack doesn't offer any tagging or labeling functionality, apart from categorization by channel. And you can't expect that all users pick the right one for their question.
  4. There is no moderation functionality — if someone posted a question in channel X and included a 500-lines code snippet, you can only politely ask to move the code to the thread and move the question to the right channel. But as a user, you shouldn't have to care about such organizational details. Why can a community moderator not control the categorization?

3) Slack gets noisy as the community grows

When a community is getting started, Slack is extremely valuable. The early users feel like they contribute to the product and help build the community. But at a certain scale, a single user contribution may be perceived as less valuable as it gets lost in the flood of hundreds of other messages per day. At some point, you may lose track of who are the top contributors and what issues are important to the community.

Imagine that you are interested in a specific feature or integration, and you would like to stay up to date on that. In a large community with no moderators governing the content and its structure, you may never find out that a discussion about your favorite feature took place because it gets lost in the noisy unstructured stream of messages.

So how does Discourse help? In Discourse, you can subscribe to a tag you are curious about. This way, you can get notified about any new topics in the area of interest. And even if you don't actively follow along with the current discussions, you can always find the information you need later.


4) Troubleshooting information not available the way you need it

Let's be honest, what's the first thing developers do when encountering an error? They google it. Slack locks the valuable troubleshooting information (that hasn't got deleted yet) within its proprietary platform, preventing this information from being found when you need it most. In contrast, the same knowledge shared via a Discourse forum is indexed by Google and is available when you google your error messages.

5) Reactivity

What if you want to share some code examples with the community proactively? This is not a pattern slack is optimized for. Remember: Slack is optimized for writing, not for capturing knowledge or reading.

Slack encourages reactivity: it makes it incredibly easy for a user to write a post and for another user to post a reply immediately. But it may not always be a good idea. Sometimes, it may be better to respond with a more comprehensive, high-quality tutorial that will provide much more context and can answer many more future questions. And this is what Discourse is optimized for, but Slack isn't.

So is Slack all bad?

Slack has different use cases and some incredibly valuable features that a forum like Discourse doesn't have (yet); it consists of direct contact that mimics a natural in-person conversation. Direct synchronous communication makes it much easier to help during urgent issues and production incidents when every minute counts.

Similarly, if a user doesn't want to share some information publicly, they can send it to you via a Direct Message. Having the possibility of a private one-to-one conversation can be helpful to users who may have a question but are too shy to ask publicly.

Lastly, chat-based communication allows you to post near-real-time announcements that would be difficult to replicate in a forum, such as:

  • "Our live stream begins in 5 minutes!"
  • "The service X is temporarily unavailable. We are working on a resolution and will keep you up-to-date."

Using Slack and Discourse in tandem

When used correctly and for the right reasons, Slack can be highly beneficial in growing a healthy and engaged community. But it should not be used for everything. Slack is an ephemeral medium; thus, it cannot be used as a knowledge carrier.

Also, not every message is urgent. Many in-depth technical discussions are better when people take more time to think about their answers rather than responding immediately. Therefore, some technical conversations can be better served by asynchronous communication such as Discourse. It encourages you to take more time to think about your answers rather than responding immediately via instant messaging platforms, such as Slack.

To prove that we fully support both tools for community interactions, we've set up an integration that allows us to transcribe any Slack conversation into a Discourse topic, combining the best of both worlds.


So…Where should I ask my questions?

We leave it up to your preference which medium do you wish to use, given the type of your question. The Prefect Community will answer all your Prefect-related questions wherever they are coming from — it may be Discourse, Slack, StackOverflow, a GitHub issue, or even plain old email.

From now on, we will work on establishing Discourse as a knowledge base, where you can find plenty of information at your fingertips (literally by typing them in Google). But it's up to you to decide whether you want to send your community support question via a Discourse topic or some other channel.

Thanks for reading, and happy engineering!