Have you ever wondered what a web server looks like from the inside? Have you ever dreamt of creating one yourself? You've come to the right place!

Welcome to this first technical article dedicated to the development of PandApache3.

This article and the following ones aim to describe the internal workings of PandApache3, a lightweight and minimalist web server ready to compete with Apache2 (We are in favor of the retirement of 29 annuities).

These articles are not documentation. They will not be updated as PandApache3 evolves. Their goal is rather to share and explain code and design choices. Some parts of the code will be simplified to be more digestible, facilitating the general understanding of our project.

Before we proceed, are you familiar with PandApache3? If not, you can learn more about it in our previous article: PandApache3, the Apache killer.

But where to begin? Describing a project from scratch is always a challenge. We will focus on the essential elements to get started, avoiding getting lost in details that might confuse beginners. Let's start by exploring the most fundamental task of a service: startup.

Liftoff: The First Steps of PandApache3

What does PandApache3 do when starting the service? Before accepting HTTP requests and listening on a port, several tasks need to be completed. In this part, we focus on the actions taken before the first connection to the service can be established.

Our startup method is called StartServerAsync, which is the very first method called when our server is launched.

public static async Task StartServerAsync()
{
    Logger.Initialize();

    Server.STATUS = "PandApache3 is starting";
    Logger.LogInfo($"{Server.STATUS}");

    ServerConfiguration.Instance.ReloadConfiguration();

    _ConnectionManager = new ConnectionManager();

    TerminalMiddleware terminalMiddleware = new TerminalMiddleware();
    RoutingMiddleware routingMiddleware = new RoutingMiddleware(terminalMiddleware.InvokeAsync, fileManager);
    LoggerMiddleware loggerMiddleware = new LoggerMiddleware(authenticationMiddleware.InvokeAsync);
    Func<HttpContext, Task> pipeline = loggerMiddleware.InvokeAsync;

    await _ConnectionManagerWeb.StartAsync(pipeline);
}

The first step is to initialize our logger. The logger is an essential class that records all actions, errors, and messages of the server. This is particularly crucial during startup, as it needs to be ready to report any potential issues, as illustrated by logging the "is starting" status on the third line.

Logging information can be available in two places depending on the chosen configuration:

• In log files, which is the classic service configuration. A PandApache3.log file is created, and each event is logged there.
• In the console, which is very useful for directly viewing logs on the console output or terminal, in addition to or instead of log files.

These two options can also be combined, allowing you to choose how to manage your logs according to your needs.

Between us

Why opt for NoLog or logs only in the console rather than in a file? At first glance, it may seem strange not to keep logs in a file. However, this decision is strategic for PandApache3, designed to be PaaS-friendly. When managing a platform as a service (PaaS) with thousands of instances, storing logs on the server can pose accessibility and disk space issues. It is therefore wiser to redirect application-generated logs from the console to a dedicated system such as ADX or Elastic Search.

This approach also facilitates quick feedback during application development.

Finally, the ability to use NoLog with PandApache3 (by disabling log writing both in the file and in the console) is a direct consequence of the flexibility offered by the service.

Diving into Configuration:

Like any service, PandApache3 is configurable. After initializing the logs, loading the configuration becomes the second mandatory step. This configuration, available in the form of the PandApache3.conf file on the machine, plays a crucial role in the behavior and functionality of PandApache3.

public void ReloadConfiguration()
{
    string fullPath = Path.Combine(_configurationPath, "PandApache3.conf");
    if (!File.Exists(fullPath))
    {
        throw new FileNotFoundException("The configuration file didn't exist", fullPath);
    }

    try
    {
        foreach (var line in File.ReadLines(fullPath))
        {
            if (string.IsNullOrWhiteSpace(line) || line.Trim().StartsWith("#"))
            {
                continue;
            }
            else
            {
                var parts = line.Split(new[] { ' ' }, 2, StringSplitOptions.RemoveEmptyEntries);
                if (parts.Length == 2)
                {
                    var key = parts[0].Trim();
                    var value = parts[1].Trim();

                    MapConfiguration(key, value);
                }
            }
        }
        Logger.LogInfo("Configuration reloaded");
    }
    catch (Exception ex)
    {
        throw new Exception($"Error during configuration reload: {ex.Message}");
    }
}

public void MapConfiguration(string key, string value)
{
    var actionMap = new Dictionary<string, Action<string>>
    {
        ["servername"] = v => ServerName = v,
        ["serverip"] = v =>
        {
            if (IPAddress.TryParse(v, out var parsedIPAddress))
                ServerIP = parsedIPAddress;
            else
                Logger.LogWarning("Server IP invalid");
        },
        ["serverport"] = v => TrySetIntValue(v, val => ServerPort = val, "Server port invalid"),
    };

    if (actionMap.TryGetValue(key.ToLower(), out var action))
    {
        action(value);
    }
    else
    {
        Logger.LogWarning($"Unknown configuration key: {key}");
    }
}

This ReloadConfiguration function loads each line from the PandApache3.conf file (excluding comments), mapping each key to a value. Then, the MapConfiguration function uses a dictionary (actionMap) to map each key to an action to perform before assigning the value to the class variable.

For example, for the line: ["servername"] = v => ServerName = v,, the dictionary key is servername, and the associated action is v => ServerName = v, where v represents the value. The action is a lambda function that assigns this value to the ServerName property.

Now equipped with the necessary information, our server is ready to start according to the provided specifications and to provide feedback in case of issues. Let's move on to the next step: connection management!

Between us

An error in the configuration parameter is not blocking; a warning will be issued, but the service will still start! In case the configuration file is missing, the application's default parameters will be used.

Still between us

Why choose a .conf file in text format instead of JSON or YAML? Firstly, for its simplicity: it's easier to write a first configuration file in text format than editing JSON or YAML, which can be problematic without a good editor. Moreover, text format allows comments, which is very convenient for self-documentation of the configuration file. In the future, supporting multiple file formats to manage configuration is not excluded.

The Heart of PandApache3: The Connection Manager

The heart of our PandApache3 server lies in its connection manager, represented by the ConnectionManager object.

_ConnectionManager = new ConnectionManager();

This relatively simple object has two key attributes: TcpListener and pipeline.

public TcpListener Listener { get; set; }
private Func<HttpContext, Task> _pipeline;

The TcpListener is a fundamental component that allows clients to connect to our server via the TCP protocol. As for our _pipeline variable, it represents an asynchronous function that takes an HTTP context (HttpContext) as a parameter and returns a task (Task). In a figurative sense, our pipeline is a series of actions we want to execute on each HTTP request. Each action is performed by what we call middleware.

In fact, in the following code, we set up the middlewares to be used for each received HTTP request:

TerminalMiddleware terminalMiddleware = new TerminalMiddleware();
RoutingMiddleware routingMiddleware = new RoutingMiddleware(terminalMiddleware.InvokeAsync);
LoggerMiddleware loggerMiddleware = new LoggerMiddleware(authenticationMiddleware.InvokeAsync);
Func<HttpContext, Task> pipeline = loggerMiddleware.InvokeAsync;

So we have three middlewares here:

• TerminalMiddleware
• RoutingMiddleware
• LoggerMiddleware

Each middleware calls the next one in a well-defined chain (Logger calls Routing, then Routing calls Terminal). This chain of middlewares (our pipeline) is assigned to our connection manager (ConnectionManager).

Now that everything is set up, we can start our connection manager:

await _ConnectionManagerWeb.StartAsync(pipeline);

The StartAsync function simply configures our TcpListener to listen on the IP address and port defined in the configuration, and then starts it:

public async Task StartAsync(Func<HttpContext, Task> pipeline)
{
    Listener = new TcpListener(ServerConfiguration.Instance.ServerIP, ServerConfiguration.Instance.ServerPort);
    Logger.Log

Info($"Web server listening on {ServerConfiguration.Instance.ServerIP}:{ServerConfiguration.Instance.ServerPort}");
    Listener.Start();
    _pipeline = pipeline;
}

There you have it, our server is now started and ready to receive incoming connections.

Between us

What the middlewares do is not crucial at the moment. What matters is that our ConnectionManager, responsible for handling incoming connections on its TCP listener, will pass them all through this chain of middlewares and in this order.
However, the names are quite self-explanatory, and you can guess the role of each middleware:

• Logger: Logs the incoming request.
• Routing: Directs the request to the correct resource.
• Terminal: The last middleware in the chain, which does nothing particular but is there.

Still between us

A request that goes through the middlewares does so both on the way in and on the way back (in reverse order). In our example, this means the request is first logged by the first middleware, and then the obtained response is also logged by this same middleware now become the last in the chain.

Thank you so much for exploring the inner workings of PandApache3 with me! Your thoughts and support are crucial in advancing this project. 🚀
Feel free to share your ideas and impressions in the comments below. I look forward to hearing from you!

Follow my adventures on Twitter @pykpyky to stay updated on all the news.

You can also explore the full project on GitHub and join me for live coding sessions on Twitch for exciting and interactive sessions. See you soon behind the screen!