Microservice first vs Monolith first ~

[Azzizz81]

Are most enterprise projects these days microservice first or monolith first ~

 

[davidktw]

Are most enterprise projects these days microservice first or monolith first ~

Why not both? It is not as if either have absolute advantage over another. Monolithic for components that are meant to be deeply coupled and requires low latency. Microservices for components that need to scale and high throughput.

Be flexible and versatile, what is the size of a service, how small is micro,; the smaller the micro, the more network interaction.

Personally I find there are so much high level talks that are just in my opinion 纸上谈兵. On the ground execution will requires involvement of business organisation and politics. In the end, a holistic design doesn’t works well with organisation governance knowing who is funding the project.

 

[Azzizz81]

https://martinfowler.com/bliki/MonolithFirst.html

As I hear stories about teams using aÂ*microservices architecture, I've noticed a common pattern.

Almost all the successful microservice stories have started with a monolith that got too big and was broken up

Almost all the cases where I've heard of a system that was built as a microservice system from scratch, it has ended up in serious trouble.

This pattern has led many of my colleagues to argue thatÂ*you shouldn't start a new project with microservices, even if you're sure your application will be big enough to make it worthwhile.Â*.

I have heard of some microservice projects that become delayed. Besides scalability, what other benefits does it bring~¿

 

[davidktw]

https://martinfowler.com/bliki/MonolithFirst.html

I have heard of some microservice projects that become delayed. Besides scalability, what other benefits does it bring~¿

Personally the read is a good read. The writer is pragmatic and he has IMO pretty much nailed it. Having an ambitious business model is important, because you need to know your general direction of scaling your business, but you start small to get traction and to be agile so to speak.

Agility doesn’t come from having things spread all over the place and then you chain them together. Agility is the speed at which you adapt. A monolithic design doesn’t mean it cannot be changed, in fact, it can adapt pretty quickly when you are starting it because you are always dealing with the center piece.

While so often implementing and dealing with clients requirements and line of thought, my general idea is too many of them think too big and too much.

There are some projects that are evolution from their predecessors, those normally have a more sane expectation because when you ask the client to give you a sizing, they can fetch it from their existing system and with some reasonable growth factor and extrapolation, you roughly knows what sizing you are dealing with.

However there are also projects which are from scratch. They are new ideas, new directions or new offerings to their end-users. But they think too big, they thought they are designing the Facebook now, or Netflix, or the latest Google Map. They assume too much into the sizing, plucking numbers from the sky and think their system is going to be welcome by all their end-users because they spend tons of time doing research i’m thinking what the end-user wants.

Before the system is implemented or actually being largely used by the end-users, they already foresee the sizing and the use cases and so forth. They thought they have set a reasonable performance and scale expectation, and ran performance test, trying to test the system to its limit. When it is finally launched, either the use cases are totally different or the load is miniscule. Basically there is no real implementation before it to ascertain the decisions made, they are all based on best guesses. But most are just too overly zealous thinking they must built the system to ready for global scale access. When you set such an expectation commercially, there is no room for error or someone head must roll, so everyone tried to play defensively and in the end you get a monster built for the minority. This is so often experienced in certain sector projects as I have observed.

Therefore I am all in for the onion approach of solving problem. First you built a monolithic system to test out the acceptance. A smaller system makes it easier to implement and to change and to operate. You set up your design in phases, and have an experimental mindset. You make changes along the way, tackling areas that are forming bottlenecks. You peel the onion on its way out, to create fan out design for components that matters. These things will go in phases until there is a point where there is sufficient real statistics to show where are the deficiencies. Then you plan for the next evolution. You now can consider a higher scale of design, using microservices, SOA, enterprise buses, queue systems and all sorts of large scale deployment techniques to ease out on operations knowing the next system must be cater for a larger scale than this monolithic design may not work anymore.

You don’t need to just stick with one methodology. It can be both monolithic and services oriented. Over the years or changing, assess on the operations, go back to the drawing board and redraw and redo. Designing system is never a one step process, and when you add in the business elements where fundings and support doesn’t come in one big pile, why have an overly design system that start off like a monster, making it a burden instead of an asset?

Recently there is a sudden surge of interest in APIs. Because we have renowned public services like Facebook, Instagram, Twitter, or some other services being successful in their attempt to round up developers to design in their ecosystem, now everyone jump into the API GW wagon. Suddenly monolithic API found in frameworks are not good enough, they must be accompanied with API GW that start off with 5 or more services running in an orchestration. They need to use OAUTH, SAML, or external sign in with Facebook, OpenID and other federated approach. But there is no API being discussed right from the beginning. There are little to no systems that are currently offering APIs. But the mindset is, since all these big companies are so successful, so we should start on on the right track. We need developer portal, we need scalable API GW able to handle thousands of concurrent connections. They need to scale affordably. The system must throttle effectively with circuit breaker. We probably will want to use GraphQL instead of just RESTful because Facebook is successful in it.

But the questions to really ask are “Why do developers or partners want to develop apps or integrate with our systems?”, “What are the incentives for them to join?”, “ What is in for them to come join our ecosystem?”, “How do we convince them and court them?” These are often the situation isn’t it? Think of the gadget first, and then think about how to use the gadget later. Sounds familiar?

 

[Trader11]

Personally the read is a good read. The writer is pragmatic and he has IMO pretty much nailed it. Having an ambitious business model is important, because you need to know your general direction of scaling your business, but you start small to get traction and to be agile so to speak.

Agility doesn’t come from having things spread all over the place and then you chain them together. Agility is the speed at which you adapt. A monolithic design doesn’t mean it cannot be changed, in fact, it can adapt pretty quickly when you are starting it because you are always dealing with the center piece.

While so often implementing and dealing with clients requirements and line of thought, my general idea is too many of them think too big and too much.

There are some projects that are evolution from their predecessors, those normally have a more sane expectation because when you ask the client to give you a sizing, they can fetch it from their existing system and with some reasonable growth factor and extrapolation, you roughly knows what sizing you are dealing with.

However there are also projects which are from scratch. They are new ideas, new directions or new offerings to their end-users. But they think too big, they thought they are designing the Facebook now, or Netflix, or the latest Google Map. They assume too much into the sizing, plucking numbers from the sky and think their system is going to be welcome by all their end-users because they spend tons of time doing research i’m thinking what the end-user wants.

Before the system is implemented or actually being largely used by the end-users, they already foresee the sizing and the use cases and so forth. They thought they have set a reasonable performance and scale expectation, and ran performance test, trying to test the system to its limit. When it is finally launched, either the use cases are totally different or the load is miniscule. Basically there is no real implementation before it to ascertain the decisions made, they are all based on best guesses. But most are just too overly zealous thinking they must built the system to ready for global scale access. When you set such an expectation commercially, there is no room for error or someone head must roll, so everyone tried to play defensively and in the end you get a monster built for the minority. This is so often experienced in certain sector projects as I have observed.

Therefore I am all in for the onion approach of solving problem. First you built a monolithic system to test out the acceptance. A smaller system makes it easier to implement and to change and to operate. You set up your design in phases, and have an experimental mindset. You make changes along the way, tackling areas that are forming bottlenecks. You peel the onion on its way out, to create fan out design for components that matters. These things will go in phases until there is a point where there is sufficient real statistics to show where are the deficiencies. Then you plan for the next evolution. You now can consider a higher scale of design, using microservices, SOA, enterprise buses, queue systems and all sorts of large scale deployment techniques to ease out on operations knowing the next system must be cater for a larger scale than this monolithic design may not work anymore.

You don’t need to just stick with one methodology. It can be both monolithic and services oriented. Over the years or changing, assess on the operations, go back to the drawing board and redraw and redo. Designing system is never a one step process, and when you add in the business elements where fundings and support doesn’t come in one big pile, why have an overly design system that start off like a monster, making it a burden instead of an asset?

Recently there is a sudden surge of interest in APIs. Because we have renowned public services like Facebook, Instagram, Twitter, or some other services being successful in their attempt to round up developers to design in their ecosystem, now everyone jump into the API GW wagon. Suddenly monolithic API found in frameworks are not good enough, they must be accompanied with API GW that start off with 5 or more services running in an orchestration. They need to use OAUTH, SAML, or external sign in with Facebook, OpenID and other federated approach. But there is no API being discussed right from the beginning. There are little to no systems that are currently offering APIs. But the mindset is, since all these big companies are so successful, so we should start on on the right track. We need developer portal, we need scalable API GW able to handle thousands of concurrent connections. They need to scale affordably. The system must throttle effectively with circuit breaker. We probably will want to use GraphQL instead of just RESTful because Facebook is successful in it.

But the questions to really ask are “Why do developers or partners want to develop apps or integrate with our systems?”, “What are the incentives for them to join?”, “ What is in for them to come join our ecosystem?”, “How do we convince them and court them?” These are often the situation isn’t it? Think of the gadget first, and then think about how to use the gadget later. Sounds familiar?

You mean Graphql should not be adopted for new project unless there are too many rest api calls that can be simplified with graphql?

 

[davidktw]

You mean Graphql should not be adopted for new project unless there are too many rest api calls that can be simplified with graphql?

You are mistaken in what I am sharing. I'm not sharing which technology you shouldn't adopt. I'm sharing why you will want to adopt a technology for needs rather than wants.

If you want to choose GraphQL, you will also want to understand what value it brings and what complexity it brings.

GraphQL is analogous to SQL, but with the flexibility of forming joins and complex queries, you are also opening up your application for potential abuse. Your implementation will need to take care of queries that may result in high cpu load and/or high memory load depending on how your backend API works.

Normally the expectation of accessing 2 different tables within a single SQL is transaction in behaviour and by default exhibit read-commit isolation.

That means for an SQL of the following

will means any other closed transactions happening before this statement will definitely be included in this SQL results sets, while other unclosed transactions happening while this statement is executed will not be included in this result set.

The database is doing a lot to create this form of isolation for consistency. Trying to formulate this at an application layer should T1 and T2 are not part of the RDBMS, which could be just data rows from a JSON file, or external API calls or NoSQL system, means you will not naturally get such consistent behaviour. Implementing it at the server side means the system commit to provide such behaviour, not unless you explicitly deny it.

Also when the database generate a dataset that may potentially contain millions of rows of results, the database is designed to handle this using mapped files, using on-disk virtual tables, and other techniques to prevent overloading the system. Offering GraphQL for flexible queries will also potentially create a venue for DOS if the user don't follow the rules properly, or your implementation failed to take care of queries that can exceed the maximum time required or maximum results size allowed. All these are taken care by the database.

So you see my point is choosing GraphQL over a simple RESTful API is not just a matter of simple choices. The potential to flip is there. If it is just blindly following the trend of systems out there which takes a lot of effort to implement it properly, but you never knows how much effort the said system has taken to protect, it's simply shooting yourself in the foot when you assumed the responsibility of the client under the server charge.

GraphQL is not necessarily better, it consolidate and alleviate your clients for the needs to create multiple API queries, but the server also undertake a much more important role and also much higher complexity that wasn't there when the RESTful API are simpler segregated calls.

 

[Trader11]

You are mistaken in what I am sharing. I'm not sharing which technology you shouldn't adopt. I'm sharing why you will want to adopt a technology for needs rather than wants.

If you want to choose GraphQL, you will also want to understand what value it brings and what complexity it brings.

GraphQL is analogous to SQL, but with the flexibility of forming joins and complex queries, you are also opening up your application for potential abuse. Your implementation will need to take care of queries that may result in high cpu load and/or high memory load depending on how your backend API works.

Normally the expectation of accessing 2 different tables within a single SQL is transaction in behaviour and by default exhibit read-commit isolation.

That means for an SQL of the following

will means any other closed transactions happening before this statement will definitely be included in this SQL results sets, while other unclosed transactions happening while this statement is executed will not be included in this result set.

The database is doing a lot to create this form of isolation for consistency. Trying to formulate this at an application layer should T1 and T2 are not part of the RDBMS, which could be just data rows from a JSON file, or external API calls or NoSQL system, means you will not naturally get such consistent behaviour. Implementing it at the server side means the system commit to provide such behaviour, not unless you explicitly deny it.

Also when the database generate a dataset that may potentially contain millions of rows of results, the database is designed to handle this using mapped files, using on-disk virtual tables, and other techniques to prevent overloading the system. Offering GraphQL for flexible queries will also potentially create a venue for DOS if the user don't follow the rules properly, or your implementation failed to take care of queries that can exceed the maximum time required or maximum results size allowed. All these are taken care by the database.

So you see my point is choosing GraphQL over a simple RESTful API is not just a matter of simple choices. The potential to flip is there. If it is just blindly following the trend of systems out there which takes a lot of effort to implement it properly, but you never knows how much effort the said system has taken to protect, it's simply shooting yourself in the foot when you assumed the responsibility of the client under the server charge.

GraphQL is not necessarily better, it consolidate and alleviate your clients for the needs to create multiple API queries, but the server also undertake a much more important role and also much higher complexity that wasn't there when the RESTful API are simpler segregated calls.

Yes it is susceptible to SQL injection as well. Also introspection can leak schema data if the dev ignore security.

I don't see many companies using this. Most likely Facebook....

But I think it is good for technologist to learn this to understand how it works. Not necessarily everything you learn need to be implemented in production.

 

[davidktw]

Yes it is susceptible to SQL injection as well. Also introspection can leak schema data if the dev ignore security.

I don't see many companies using this. Most likely Facebook....

But I think it is good for technologist to learn this to understand how it works. Not necessarily everything you learn need to be implemented in production.

Sure learn it. Learn when to use it and when not to use it