Preamble

Planning to shift to Microservices? Before you make the leap, let's discuss a few key things to keep in mind. Considering these factors upfront can make your transition smoother and ensure you maximize the benefits of Microservices.

Maintenance Cost

When your team decide to go Microservice, you will need multiple compute engine where your service put on. In my opinion for this problem you can go with Domain-Driven-Design concept, it teaches you not to overslice your big service into "Too Much Services" and it show you ways how to categorizing service based on its busines domain, and decrease amout of compute engine should be prepared for your services. Ofc it affects your cost to spend on compute engines. Here's an example for Domain-Driven-Design.

Scaling Factors

Before moving to Microservice you should check your service, is it Stateless or Stateful service? because it help you to decide using Horizontal or Vertical scaling?

What is Stateful and Stateless Service?

Here is a illustration of Stateful and Stateless in simple way.

• Stateful is a behavior of a System or you can said application that holds bunch of retained data to supports its functionallity.

• Stateless is a behavior of a System or you can said application that does not holds retained data to supports it functionallity.

What is Horizontal and Vertical Scalling?

Here is a illustration of Horizontal and Vertical Scalling in simple way.

• Horizontal Scalling method is, Increase amount of Physical Device or Cloud Instance, where your service will be put on.

• Verticall Scalling method is, Increase Compute Power of existing Physical Device or Cloud Instance, where your service will be put on.

Stateful and should have Load Balancer?

If yes, in this condition your service should not use Horizontal scaling, because every instances for Load Balancing purpose will be state inconsistent. And it will return different output while Load Balancing choose which service should be burdened next. If this condition inevitable you need to consider how retained data of each service synchronized around instances.

Stateful and should not have Load Balancer?

If yes, in this condition your service could use Vertical scaling, because when the service get stressed, instance will automatically increase computation power.

Stateless and should have Load Balancer?

If yes, in this condition your service could use Horizontal scaling, because there is no retained data inside each service on the instances, and each service will return consistent output everytime the Load Balancer moving to next service.

Stateless and should not have Load Balancer?

If yes, in this condition your service could use Vertical scaling, because there is no retained data inside each service on the instances, and when the service get stressed, instance will automatically increase computation power.

Data Source Strategy

Choosing which Data Source Engine should be used is a crucial decision. You need to address the problems your business might face, whether to prioritize performance over consistency or vice versa. In our development journey there are two Data Source Engine which are popular, there are NoSQL and SQL. Let's simplify, go with NoSQL to prioritize performance, otherwise go with SQL to embrace consistency. Or you can mixed it up for flexibility. After you have decided let's talk about data source strategy for your service.

One Database for All Service

In a microservices setup, "One Database for All Service", means that every service you have, uses the same central database to store information.

Advantages

• Simplicity: Managing a single database can simplify the overall architecture, as there's no need to deal with distributed data management or service-specific databases.

• Consistency: Since all services share the same database, maintaining data consistency and integrity can be easier compared to having multiple databases with potential data duplication or inconsistency.

• Data Sharing: All services have access to the same dataset, enabling easier data sharing and communication between services.

• Cost Savings: Managing a single database may be more cost-effective in terms of infrastructure and maintenance compared to multiple databases.

Challenges

• Scaling: Scaling can become challenging as the system grows, as all services share the same database resources. Performance bottlenecks may arise if the database becomes a bottleneck.

• Dependency: All services become tightly coupled to the shared database schema, making it difficult to make changes to the database schema without affecting multiple services.

• Data Ownership: It can be unclear which service owns which data, leading to potential confusion and conflicts in data management responsibilities.

• Security: Access control and security concerns become critical, as any security breach or data leakage in the shared database can affect multiple services.

One Database for Each Service

In a microservices setup, "One Database for Each Service" approach means each service you have, has its own database to handle its data.

Advantages

• ⁠Isolation: Each microservice has its own database, promoting isolation and independence. This isolation reduces the risk of data coupling between services and allows for more straightforward database schema changes without impacting other services.

• ⁠Scalability: Scaling becomes more granular and flexible since each service can scale its database independently based on its specific workload and performance requirements.

• ⁠Flexibility: Different services may have diverse data storage needs, such as relational, document-oriented, or key-value data stores. With separate databases per service, teams can select the most appropriate database technology for each service's requirements.

• ⁠ ⁠Security: Data isolation enhances security by limiting access to each microservice's data only to authorized users and services, reducing the risk of unauthorized access or data breaches affecting multiple services.

Challenges

• Data Consistency: Ensuring data consistency across multiple microservices can be challenging when related data spans multiple databases. Implementing distributed transactions or eventual consistency mechanisms may be necessary to maintain data integrity.

• ⁠Complexity: Managing multiple databases introduces additional operational complexity, including backup and recovery, monitoring, and schema migrations. Teams must have the necessary expertise and tooling to manage these databases effectively.

• ⁠Resource Consumption: Running multiple databases increases resource consumption, including storage, memory, and CPU usage. Proper resource allocation and optimization are essential to avoid resource contention and performance degradation.

One Shared Database for Each Service

In a microservice setup, the "One Shared Database for Each Service" approach means every service you have has its own database, but other service also can access others service database.

Advantages

• Isolation and Independence: Similar to the "One Database for Each Service" approach, each microservice retains its database, promoting isolation and independence. This separation allows for individual service teams to manage their data schemas and operations without affecting other services.

• Resource Sharing: By sharing the same underlying database infrastructure, organizations can benefit from resource pooling and optimization. This can lead to more efficient resource utilization compared to managing entirely separate database clusters for each service.

• ⁠Consistency and Compatibility: Since all services interact with the same database schema, maintaining data consistency and ensuring compatibility between services becomes more straightforward. Changes to the database schema can be applied uniformly across all services, reducing the risk of schema mismatches or compatibility issues.

• ⁠Operational Efficiency: Managing a shared database infrastructure simplifies operational tasks such as backup and recovery, monitoring, and database upgrades. Organizations can leverage centralized tools and processes to streamline database management across all services.

Challenges

• ⁠Data Coupling: While each service has its database instance, they are still coupled at the database schema level. Changes to the database schema may impact multiple services, requiring coordination and communication between service teams to ensure smooth migrations and updates.

• Performance Bottlenecks: Introducing a shared database infrastructure may lead to performance bottlenecks, especially if certain services generate high volumes of traffic or have complex data access patterns. Careful monitoring and optimization are necessary to prevent performance degradation.

• ⁠Security Concerns: Sharing a database infrastructure raises security concerns, as unauthorized access or data breaches in one service's database could potentially impact other services. Robust access control mechanisms and security measures must be implemented to mitigate these risks.

Communication Style

There are two popular communication styles: REST API and gRPC. Both have similar concepts but differ in output result. While REST API remains powerful and widely used, it may not be ideal for communication between services. It is more suitable for communicating with customer-facing systems due to its use of string-based responses, which are easy to serialize. On the other hand, gRPC is an excellent choice for communication between services because it uses bytes as the output, which is less CPU intensive to consume, aiming for better data exchange performance. However, gRPC may not be ideal for communication with customer-facing systems.

REST API

REST API, while still powerful and widely used, may not be the best choice for communication between services. It is more suited for interactions with customer-facing systems due to its string-based response format, which is easy to serialize. REST API offers you rapid development because of easy to implement and its popularity.

gRPC

gRPC is a great choice for communication between services, offering better data exchange performance by using bytes as the output. However, it may not be the ideal option for communicating with customer-facing systems. gRPC might cause development overwhelm because its tricky to implement and many steps to follow to make one data model.

Distributed Tracing

When decide moving to Microservice some of you might forget implement Distrubute Tracing to all service, and bottleneck becomes hard to find, without distributed tracing, and you need to research about how to use distributed tracing, for handle this problem, and you need to serve the information about your service to distributed tracer by adding event in every repository function and business logic function, to trace out the bottleneck, you can use whatever you want, OpenTelemetry is the one I recommended to you.

Contextual Logging and Logstash with Kibana

This one is a must, every activity from incoming request until the service response to a request, it should be stampled with "Request Id" or you can use other terms for that, this one called Contextual Logging where you can trace execution flow of a business logic request only by Request Id of a request, Logstash with Kibana will be very good choice for you to implement, it helps you to wrapping up all Logging Files content into Logstash and feeded to Kibana, and you just need to use the Query in the Kibana Dashboard, you will find execution footprint accross the service, if you implement it properly.

Conclusion

In conclusion, before making the move to Microservices, it's important to weigh whether your service is stateless or stateful, choose between horizontal or vertical scaling, and consider reducing maintenance costs through Domain Driven Design categorization. Additionally, setting up contextual logging and Logstash with Kibana, along with implementing Distributed Tracing, can greatly facilitate debugging across services. And for Data Source Strategy in this topic provides 3 strategies, you can define your own strategy with better consideration and best for your service, those are just example base on my experience. Communication Style, REST API or gRPC its depends on your goal, and rapidity of development phase, you can go with both of them, if you want separate gRPC for communicate between service and REST API for customer-facing system, and you can still use REST API for all communication. By addressing these considerations, you can ensure a smoother transition to Microservices and enhance the overall efficiency of your system.

My Thanks

Thank you for visiting! I hope you found it useful and enjoyable. Don't hesitate to reach out if you have any questions or feedback. Happy reading!