Observability, Monitoring and Logging

0:00 - Intro

00:00:34 - What is Observability?

00:03:39 - Practical examples to make application observable.

00:04:38 - Beenefits of logging for web aplications when it comes to debugging and UX enhacements

00:06:56 - Why is discussing observability relevant to frontend architecture?

00:09:10 - In essence, observability aspect has moved to frontend.

00:10:09 - "Everything is effectively given by the machine"

00:10:27 - How can observability practices help in identifying and resolving performance bottleneck in real time?

00:15:21 - What specific challenges do composable architects face in terms of observability and how can they be overcome?

00:19:14 - The challenges of tracing system issues

Observability, Monitoring and Logging

Hello. Welcome back to the Goodfrontend show. My name is Karl, and today we're going to be talking about observability monitoring and logging.

Let's get to the first question. What is observability?

Well, I would say is when people think about observability in a technical sense, observability is a way for you to understand what is happening internally in your system through external outputs.

That's like the out of the box definition. In simple terms, observability is the ability for you to know what's happening in your system.

It helps you understand the health of your system so that you can optimise things for better debugging, for issue resolution, for tracing, for issues, and then improving user experience as well.

Imagine perhaps observability is a lot like understanding your system in a laboratory or in a more day to day example maybe is it's like putting sensor detectors and cameras in your home so that when you are away,.

You know what's happening in your home and it might provide you real time alerting and session replays a lot like ring doorbells and cameras and so on when you are not in the house so that you can see things.

Or perhaps in the family sense, if you had children at school, you might want to know how your children are doing.

There are two ways to do that.

One could be feedback, but feedback you only get maybe during your teacher and parent meeting days.

Observability is more understanding how your child is doing real time.

You'd be able to know if your child is doing well straight away if they are failing in school.

Imagine if you had this ability for you to just know straight away when they are struggling with a subject.

And that is effectively what we have with technology, with observability tooling, we are able to understand straight away when the system is not healthy or something is wrong with it or something needs improving.

Can you provide practical examples of how you can make your application more observable?

Sure. When people talk about observability, depending on which perspective you're entering, people have a different understanding of it.

So, for example, people who are working in operations and infrastructure, they would understand observability from application performance monitoring perspective, things like New Relic or Cloudwatch and infrastructure and operations related items.

From a developer perspective, they would understand observability through logging and the logging tools that they would have to be able to debug their system from a data analysis perspective.

They think of observability from data analytics and then from UX and product design, they would think of observability in the sense of user experience analytics platforms like Contentsquare.

So these are practical examples when we're thinking about observability so holistically,

Observability would be all the way infrastructure related stuff relating to New Relic to the application itself in relation to logging, to the user experience and user behaviour using UX analytics platform.

And the performance of the commercial side using Google Analytics and tools similar to that.

So in practical terms, you've got things like Contentsquare, Google Analytics, New Relic, Kibana, you know, the Elk system, Elasticsearch with Logstash and Kibana.

Or the the tools that come with the infrastructure providers like AWS, GCP, or if you're using something on top like Netlify and Vercel.

How can logging be beneficial for web applications in terms of debugging security and UX enhancements?

I think there's three parts to that. Let's go with debugging.

First, debugging for you to be able to understand what's happening in your application, you need your application to speak.

Logging is the way for your application to speak back to you.

So logging is really important for just understanding what's happening.

The idea is that you have different levels of logs, whether this is an information log, or a warning log, or an error log or a debug log, and by having different levels of logs, you can then drill down on the issue that you are experiencing and trace the events that are happening.

So that's one thing for debugging.

The other one is more from an operations perspective is security.

So you want to be able to understand what exactly happened.

You know, perhaps there might be a security breach you need to be able to understand, okay, how did that happen?

So it can be something that is retrospective and logs are important.

It's a lot like, you know, the black box in the plane.

When a plane crashes, the first thing they try to look for is the black box so that they can understand, okay, what exactly happened when people were on that plane.

And of course, it can also be something that is reactive with monitoring tools nowadays when maybe an attack is happening, you're able to log straight away.

Okay, where is this coming from?

You know, and so that you can respond to security incidents fairly swiftly.

But the other item about logging that is probably helpful to note, which if you're from an operations perspective is not very obvious, is the UX side of logging.

You need to be able to log the user behaviour so that you know how to improve the user experience of the website.

Again, logging, it's important to think of logging holistically all the way from the operations to the user behaviour and to the commercial aspect that is attached to that.

Why is discussing observability relevant to frontend architecture?

That's a great question.

Observability was not traditionally a frontend problem, so it's resided on operations mostly and infrastructure and maybe backend engineers would be more focussed on that.

But today much of observability really moved to the frontend because frontend has to think about the holistic experience and what what the frontend realm today is effectively the whole application.

So if you think about it, a lot of services nowadays are *-as-a-Service, so we don't even bother with checking database performance, for example, because it's not our database.

We're using a service.

A lot of the things that we are using are from cloud. So the infrastructure is as-a-Service, especially if you're using something on top of AWS and GCP and Azure, you know,.

Those more specialist Infrastructure-as-a-Service providers like Netlify, Vercel and things like that.

What tends to happen is that it almost like thins out the infrastructure requirements and so it's not really worth it for businesses, say, to create specialisations for those areas because it's very little work to do.

And so what happens is that it just gets rolled on as part of the software engineering role.

And so understanding observability and monitoring and logging and all the things that are related to this operation side of things is really crucial for frontend architects because for frontend architects today,

Not only do they care about the browser side of things, but also that thin server layer using typically Node, you know, that powers their website and a lot of that.

You won't know how to deal with it unless you make that layer of your application a lot more observable.

So in essence, this observability aspect that is traditionally a problem for more the database engineers and the backend engineers have now moved on to the frontend,.

Which makes it an important aspect as part of frontend architecture.

Guess just thinking about it, it cuts down time.

Back in the day you would have to go through the database engineering or the backend engineers to find out that problem, isn't it?

But if we have this as a frontend engineer, then it would just cut down the people that you have to go through.

I think you're absolutely right because much of the log instrumentation and so on actually comes out of the box already.

So if you were using AWS, it comes with Cloudwatch and everything is already logged there.

If you were using other providers,

I was really surprised how much out of the box things are that comes with, you know, with cloud providers.

So everything is effectively given to you by the machine. You know, you no longer need to go through people,.

But the machine will tell you and they will present it to you in a better way as well, because they will provide you graphs that will allow you to understand what's happening in the application.

Can observability practices help in identifying and resolving performance bottlenecks in real time?

It's very common for enterprises to perform load tests, and the reason why they do that is because they have to handle peak times.

For example, during Black Friday, during sale periods, during, you know, season changes and so on.

And so observability really helps you understand how your application is used by your users.

Again, let's think about it in in three slices.

One is on an operational level. So from an observability operations perspective, when your users are using your application, your website,.

The observability aspect will enable you to understand if you are giving enough resources to your system, it enables you to optimise your and tune your application better.

In relation to how much memory does your application need? How much processing power does it need?

Because there's two scenarios here.

You either under provision your application and it crashes during peak time or you over provision your application and you are wasting money because

You don't even need that much processing power and memory to begin with.

There is a cost involved in both.

One is your application will fall flat and you don't want that to happen.

And the other one is that you are just throwing money, paying for more processing power and memory when you actually don't need it. Right?

So that is what observability allows you to understand how many people actually and how the people use your system in different periods.

So then you can prepare for those different periods and to be able to optimise for cost as well.

So operations really in my view is all about cost.

The other aspect of course of this observability is the user behaviour during again, when the users are using your website,.

You want to be able to understand why is it that people are dropping off at checkout?

And the traditional way that people would have done was to do conduct user testing and they would bring people in into a room and they would ask them to do a bunch of tasks.

But this is very synthetic. You know, it's not really a way to simulate proper user behaviour,.

But what a proper observability tool will allow you to do is to do session replays of the behaviour of a user using your website, you know,.

So you can perhaps segment your users, you can say, okay, from this country, from, you know, from this demographic, let's have a session replay of how they use it.

And what observability tools give you is they sort of create a picture of how this user browsed your website, how they moved from the product detail page all the way to the checkout.

And it also shows you what might be the reason that they might have dropped off.

For example, they might have gone to the checkout and you see them constantly clicking on the continue to payment, but it's not working.

You won't know that unless you have observability in place.

It's a lot like having somebody watch someone use the system, but in a more natural manner.

And then of course, the other one is when it comes to errors, right?

So let's say you've gone live, you've done your load testing, everything goes well, you're observing your user behaviour and then you're optimising the user experience.

But then there would come periods perhaps whereby all of a sudden.

Things are slowing down, right? And performance monitoring will allow you to try to understand.

Okay, when did it slow down?

Is that relating to a point in time where we made changes and so it allows you to trace and understand the issue a lot more closer.

And it will also give you which parts of the system exactly is taking a lot longer to to do things, You know,.

So I would say that really this monitoring of observability tooling is essential when setting up this web applications.

Specific challenges do composable architecture or technology face in terms of observability and how can they be overcome?

We move to cloud, there is this view that, okay, we need less infrastructure here, we need less operations people because the cloud does the stuff.

In part that is true, but that also requires people, you know, on the frontend layer to upskill, to understand that the roles are no longer just about the web browser,.

But also the layer that powers that HTML, CSS and JavaScript aspect.

So that's one part is that the move to Cloud Technologies is creating this skill gap which needs to be filled.

The other one is when it comes to microservices, is that you are distributing your systems and the challenge of that is that now you have a distributed system.

And you need to know what's happening to your system and which part of the system is going wrong.

So imagine if your user experienced an issue right, before in a monolithic application.

It's a lot easier because you've got one box and everything is in that box and you observe that box.

You know, you put logging everywhere and eventually you'll be able to trace it.

The trouble with distributed systems is that you've got multiple small boxes.

And so understanding which part of that system is going wrong becomes quite difficult because you need someone who is overall in charge.

And so this is where things like log aggregation comes into play because you need to be able to have all of these tiny little boxes that are distributed everywhere to be fed into a single log system.

And so you can see, you know, what the errors are, what's happening, where could have it gone wrong.

So that's that's one of the challenges for microservices. Distribution comes with its own set of problems and people need to realise.

That at some point you need to aggregate things for you to be able to understand what's happening in this distributed system.

So microservices has challenges.

Cloud Technologies has challenges due to skill gap, but also with cloud technologies is that it's a closed box.

You are heavily reliant on the observability of their system, which to be fair, some SaaS providers are very good at, but some SaaS providers are not very good at.

So it's a criteria that people ought to have when they are selecting this software as a service providers,.

Depending on how frequently things can go wrong, they need to be able to observe it carefully with headless.

Equally, there are challenges when it comes to observability.

Because with the headless, as I mentioned, is that frontend developers are not used to this.

This type of. The system speaking to you.

They're typically used to the system speaking to you by looking at the screen and they can see, okay, know, that's the picture that I developed so I can see it.

But there has to be a change of mindset in terms of understanding that the way you communicate with the system.

Is not only via visuals, but also via logs and understanding those different log levels.

So microservices and APIs also, you know, with API logging, it's crucial to understand where the applications are going wrong, especially because we now have caching layers.

On top of an API and then on top of our rest API, we have a GraphQL API on top of it, you know,.

So the, the amount of systems that you're having to check, which makes the tracing of issues slightly more challenging, you know, things might look like,.

Okay, it's this systems fault, but all it does is that it's just showing symptoms because it actually can't speak to another system.

So that's the reason why we talk about observability as part of our podcast is because it's an area which I notice that people are struggling with frequently.

And often they don't realise that they are struggling because they didn't know that they had to implement so much of observability in the first place.

And so observability really in modern day architecture has to be reiterated.

This is something that engineers have always known.

But as we transition, as certain boxes move towards the frontend,.

The frontend needs to understand.

That this is now their responsibility.

And that they need.

To implement it.