Transcription
Welcome, uh lovely people. Good of you to join us uh today. Um, we'll be doing a little webinar, or we have been doing a series of webinars so far. Uh, today on the schedule is uh assessing barriers or assessing controls in your uh critical control management program. For example, um, we'll do a little bit of a presentation. Uh, of course, if you have any questions, or if you're struggling with something and you would like to uh uh to raise that discussion with the group, I think there should always be opportunities for that. So please feel free to either raise your hand in the application or just speak up. Uh, that's that's perfectly fine. You can do that, just interrupt me throughout the presentation, that will be fine. I might tell you that uh I'll be covering that later in the presentation, so then we'll move on. But otherwise, you can also uh keep your questions at the end, whatever it feels more comfortable to you. You can uh also write something in the chat and uh Isaac, if I may ask you to keep an eye on the chat and alert me if anything happens, because sometimes I don't have my eyes on every single uh window in screen. Uh, bit of a cough today. Uh, so I'll, if I'll be coughing, I'll try to mute myself not to blow up your earphones, but uh uh just be aware of that. Uh, and I do apologize in advance for it. Uh, let's uh share my screen so you can actually see what's going on. Good. Uh, is my screen visible and then the slideshow, presumably too? Excellent. Great.
Um, so let's get started with this. So, as I mentioned, we'll be focusing uh on the uh assessing of the critical controls in uh the critical control management process. If you have been part of the uh webinars in previous sessions, then you, we've already discussed a little bit of the overview of what those critical control management tries to do. In a nutshell, it's sort of this kind of a life cycle, as you can see it on the screen right now. So it typically starts with some form of a risk identification process. So, what, what are my risks? Which of these risks would I'd like to include in critical control management? Uh, then typically we uh uh analyze uh those risks in more detail, and we often use bow ties for that, although you can also use other methods, of course. But the main idea is that we want to identify the different scenarios that can cause us to lose control over a particular risk, or if we can, how we can lose control over the activity that we're doing. And then specifically identify the controls that safeguard us against those situations.
Uh, now, typically, if we are creating this bow tie, we might end up having maybe 40, 50 controls on an entire diagram like that. So that's quite a lot. Uh, so typically, what most organizations do is try to identify which of those controls are really the most essential, like the really key controls, or the critical controls in that process. And then, if you are very selective, you typically end up maybe having two, three, maybe four of those critical controls in uh in your bow tie. And the idea is that once those critical controls are well under control, we have a pretty good grip on that uh uh information, uh on on that risk. And then we can also uh maybe actively try to monitor those controls. Uh, but in order to monitor the controls, or measure those controls, or basically assess those controls, we first have to have a definition of that control. So, what do we actually mean by it? Because if we don't have a good understanding of the definition of that control, it becomes very difficult to measure, because what are you measuring in that case? Uh, so the critical control definition. After that, we go into the measurement of that control. Uh, we evaluate that control. So, when the measurements come in, we subject it to some form of an analysis, or we decide with a group of experts, like, what does this mean for us? Do we want to do something about this? Uh, should we improve this control? Uh, and if we decide to do that, then of course, we can decide to improve that control, maybe fortify it, or we can also come to the conclusion that we actually don't need this control at all, and we should actually remove it and uh uh to free up resources for other controls that might be more important.
Um, and uh after that, we continue into that loop again. So that we continuously measure, evaluate, and improve if necessary. Uh, for this talk, so we'll be focusing mainly on this center part here, where we define what the control is and measure the control. The other stuff, of course, is also interesting, but yeah, we have to scope the session somehow. And this is uh the current topic IC uh of of discussion. Even these two nodes are quite a large topic in and of itself, so I had to uh condense it a little bit and try to make it uh as practicable, practical as possible. Uh, so what I tried to do is uh, yeah, design this um presentation a little bit around these four questions. So, when should we measure our controls? Like, at what point in the life cycle of your control should you be measuring it? Should you, we always be measuring it when you're executing it, or is it also okay to measure them a little bit after that, or like retroactively, like look at the control and see how it performed? Or for example, should you always measure the control even before you start executing, so you can make sure that the control is in place when you actually need it? Uh, that's an interesting uh perspective, I think, when it comes to measuring. So we'll look at that. Uh, we'll also look at uh what to measure. Um, because a control can be quite a large cluster of activities that need to happen simultaneously or in coordination with each other. So how do you decide what we should measure in a control? We'll dig a little bit deeper into that. Um, then of course, there's also a question of how we will measure it. So, what kind of data do we want to collect on our controls? There's of course all kinds of different options there. Um, every control in a way is unique. So uh uh therefore, also usually the measurements of those controls can also be unique, but they don't necessarily have to be. So we'll, we'll look at some of the options there, and perhaps you have some some practical takeaways there as well. And then the last question, uh, but not least for sure, is uh, should we actually measure these these things? Uh, should we measure all the controls in our at our disposal or not? So, like, just deciding which controls to measure is quite an important one. Uh, because you can, uh, with, yeah, with critical control management or measuring controls, I mean, you can open the door to uh uh endless measurements, basically, which is something we try to avoid. So we want to be selective. So how do we decide when to measure something and when it is also okay not to measure something? So those are the four main questions that I wanted to dive into today. Uh, that's that's definitely not everything that you can uh uh not every perspective that you can look take when measuring controls, but it's a start, and we have to start somewhere.
Um, so first one, oh, yeah, before we, I guess we go into this, uh, it's good to sort of take a step back and think about what the goals are for our uh control measurement or control assessment, basically. Um, and I think overall, if we're doing something with risk management or with safety management of some sort, the overall goal, I, in my opinion, should usually always be trying to create a safer organization. So an organization that is safer to operate in. Um, which hopefully will translate in lesser, in fewer incidents, for example. But yeah, a safer organization in general. I mean, we can dig into this definition of what actually safe really means, but I think just in general, people have an intuitive understanding that a safer organization is is probably good to work in. Um, and if that's the overall goal, well, then there's different ways, uh, yeah, different goals, sub-goals, basically, that we can take for assessing our control. So one of them could be, uh, we want to assess our controls because we want to improve the control performance over time. So, over the course of the next couple of years, we want to see an improvement in how well our controls are functioning. Uh, and in order to measure how well our controls are functioning, we have to, yeah, we have to measure those controls and compare the results, of course, over time. So we want to see, hopefully, a progression or an improvement. So that can be one goal that we have.
Another one would be to ensure that controls are always in place before work starts. That's uh uh a little bit more challenging, perhaps, but it can definitely be done. So uh that idea is, of course, that if we want to limit incidents, ideally, we want to make sure that every time we do a certain operation or a certain activity, then we want to make sure that before we commence that, all the controls, or at least the critical ones, should have should be in place uh straight away. Um, that's also a a goal, but it's of course slightly different than the first one. Uh, and both of them can, yeah, you can, you can set both of these goals, but it will uh probably affect the way you should assess your controls, and we'll, we'll take a little a little bit of a look at what the implications of that are. And then the third one can also be interesting. Uh, is more in the sense of operational decision making. So, if we know what the current status of our controls are, then we can perhaps decide to proceed within operation, or maybe hold back on an operation. So, if we know that some of our controls are impaired, then we can use that information to decide, yeah, should we now be doing this operation, or should we maybe postpone it a little bit? So, if we know that not all of our controls are working as well as we would like, do we want to send this person up on that scaffold to work at height, for example, or do we want to operate in this this manner at this time? In order to make those decisions, you also have to have uh, yeah, barrier assessment information or data on the performance of those controls. But it might be slightly different again. So, even though these are three goals that we can can set ourselves for assessing controls, um, each of these goals will require probably slightly different different types of information, and I'll try to to sort of uh keep that in mind as we go through the presentation. Does that make sense so far? If not, feel free to to speak up. Otherwise, I'll just move on.
Um, yeah, so the main takeaway of this slide, I would say, is think about what do we really want to achieve with our measurement of our controls. Uh, because it will influence the way we we should measure them and what kind of data we're collecting. So, uh, as I uh mentioned before, like in this little graph, this presentation assumes that you've already made your bow tie and you already know what your critical controls are. So we are already at a point where, like, assessment has been made on your risk, where you know what the scenarios are, you know what the controls are, and you've even already highlighted some of these controls that you feel are critical. So this image, for example, the dark purple ones would be considered the critical ones, probably. Uh, I think we had a previous webinar where we focused on a little bit more on that aspect, so how do you identify what what is critical, but uh at this stage, we assume that it is done.
So then the first question was like, when in the barriers life cycle do we want to uh measure our controls? Uh, and to try to make that practical, I, I thought I thought of envisioning it like this. So here we have this little block represents the uh the control in the moment it is needed. Uh, and then you have sort of a time span in front and and behind it. So uh uh this is the moment in time this uh control is being executed, and then there is a little bit of a time period afterwards and a little time before it. And anywhere on this timeline, we can try to measure the performance of this particular control. Uh, and uh you can imagine that for most of these controls, we'll be of course be repeated very often. So rather than thinking of a straight line, it can actually be a sort of a loop. Like, for example, if we're doing working at height, there are controls uh there, and every time we go working at heights, we have to go through this timeline again. Uh, so it repeats itself. Uh, uh, of course.
Um, so there's a couple of different options there. I would say all of these are, I think, in a way valid, but they are, they have implications if you go for them. So, um, one option might be, for example, to, well, try to measure it as it is being uh uh executed. So maybe we can uh gather data while the control is active or while the control is being used. Uh, the nice thing about it is, if we focus very closely on the control itself, is that we can capture very accurate data. So if we would have, for example, sensors uh checking if the control is working or not, uh, well, that can give us very direct information on whatever the the control is being used or not. But uh the difficulty with sometimes doing this is that, well, it's not always possible. And by uh introducing measurements during the execution of the task, we can sometimes introduce additional workload on people that should be focusing on actually doing the tasks rather than measuring the the safety mechanisms. So typically, if we want to measure something during the execution, we're limited to the types of measurements that don't require too much human intervention. Like, if you're, if we're uh out working in height, let's say we're we're dangling somewhere on the scaffold, we don't want to be filling out any checklists, obviously. So those kinds of measurements obviously would be out of the window. But there's typically other types of data that we can capture perhaps in that sense. So if we had sensor data, for example, that doesn't require you to fill out forms, that might be an option. But of course, not every control lends itself to that. To that situation.
Um, well, uh, you can also say, uh, well, we want to measure it post uh execution. So after we've come down from the scaffold, then we can fill out perhaps a form like, how well do we feel that this was working? Um, again, uh, this means that you, if we, if you measure, measure post execution, that means that we have to do it every time we do this, we have to fill out some kind of a measurement. Um, and if we're again using a checklist or some kind of a form, that can be very tedious, especially if this this kind of an activity happens quite often in your organization. If you have to do it once every day, or maybe multiple times a day, then you can see that people will very quickly get fed up with this kind of uh assessments. So again, the more frequent probably we want to measure things, the the the more impact it will have on the uh the work that people will have to do, like the more time it will take away from uh from their activity. So maybe in this case, also, we want to focus a little bit more closely on data collection that does not require uh a human input at that at that stage.
Uh, other people argue that we should always like try to measure it pre-execution, because that's the only moment where we can say, say, okay, we stopped because the control is not yet in place. Uh, I think uh this this typically works best for the types of controls that are uh more behavioral or require a human intervention. Like, like let's say, for example, like wearing a uh full rest harness when working at height. Uh, yeah, you can check that pre uh uh execution. Uh, so that you make sure that every time we work at height, we have this kind of equipment on board. Again, because it's around this execution, it does mean that that these kind of checks will probably have to happen quite often if we uh if you want to go that route. But it is definitely an option.
Uh, we can also uh take a slightly longer types of review cycle. So maybe every quarter, or maybe even like every two years or something like this. Uh, if we do this, the execution has already happened, so we can't really take benefit from the measurement uh beforehand. In this case, of course, but these kinds of measurement typically happen to try to improve the quality of the control over time. So these quarterly reviews or uh uh by-annual reviews or two-yearly reviews, uh which uh uh are still useful, but they will uh only benefit the goal that we had specified before, like the first one of trying to improve the the performance over time. Uh, for the uh the second one, obviously, we would have to focus more on the uh uh the the moments in time beforehand. Um, and for the operational decision making, again, we'll probably want to uh sit somewhere between post and and quarterly review. So, if you want to make operational decisions, the data has to be relatively fresh. It doesn't have to perhaps be fresh from second to second, but maybe from weekly to weekly basis, or maybe a monthly uh uh time interval is necessary. Otherwise, you're working with uh well, two years old data, for example, is not really going to uh be beneficial from an operational point of view. Um, and even quarterly might be a little bit uh uh overdoing it. So maybe you're looking there at a weekly, by-weekly, or uh uh monthly, for example, measurement. If you want to, but if you want to make sure that people wear it before they start to work, obviously we're more confined to the the uh the left hand side of the control execution mod.
Um, yeah, the preparation. Yeah, it's in a way a bit similar to the ones that are further removed here. Here. So, we can perhaps try to look at preparatory uh activities that need to take place for the control to work. That might also be an option. However, the further we are, of course, removed from the actual execution point of the control, the one here, the data typically becomes, like, after two years, can you still remember how the control is working? Two years before, I doubt it, for example. So, the further you are removed, the the poorer the quality typically of the data will become. But the nice thing about uh the the longer cycles is that it has less of an impact on your uh uh workers, because you don't bother them every single time. You just take a moment, for example, every quarter, every month, to to to check in how was this control working. So generally speaking, the closer you take the moment towards the point of execution, the more frequent you will have to measure these things, and the more of an impact you'll probably have on uh the the the work time people have to spend measuring these kind of things. If we're moving further away, then we typically uh do it more infrequently, which reduces the impact on the uh on on the person executing the job, which can be efficient. So these are some of the, yeah, some considerations about when in the controls life cycle should we uh measure it. And again, all of these are valid options. I think they will just produce different types of data that will be uh beneficial for different goals. So if your goal is to, yeah, improve the control quality over time, then it can be perfectly fine to just measure it every quarter, and that's perfectly fine. If you want to make sure that they, that the control is active every single time we do working at height, let's say, and then obviously you're tied into more of the pre-work or maybe during execution types of uh measurements. Um, and that will impact the way you can measure these controls as well, because we don't want to bother people with endless questionnaires, probably.
Um, some, yeah, I hope that's that's that's helpful. So, yeah, think about when we're measuring the controls, like at what point in the life cycle of the control do we want to measure it, and uh how can that practically be done?
Then, of course, the question is, what should we measure? So, even if we decide on a point in time, we still need to figure out like, what do we want to measure in that case? And it can often, easier, be easier said than done. Uh, once we're identifying a control, it can be a very short description in a box, box of text, but there's actually quite a lot going on to make sure that the controls actually functioning out in the field. And one way uh to identify what we consider this control to be, and uh identify what the requirements, for example, for this control might be, um, we have developed a little bit of a workflow or a little tool that I'd like to share with you right now, and it looks a bit like this. It's a bit of a chart. This is the empty chart. We'll look at the example in the next slide, but typically on top, you will write the the name of the control. So, for example, like your fall arrest harness, or something like this. And then you'd also like to describe the control function, and that's basically what the control, like a description of what the control really aims to achieve. And we'll, we'll take a look at that, what it looks like in a minute. Um, and then for each of the controls, we can break down uh the operational requirements. Those are basically the steps or the uh the actions required on the day of execution. Okay, no problem. Thank you. See you next time. Oper. So the steps are required for the control to to function on the day of execution. So let's say you arrive at the site, what steps need to happen on the best day, for example, for this control to work? So typically, if we do this with the clients in workshops, we uh we do a bit of a brainstorm, like, uh on an ideal day, when you have to work with this control, what would it look like? So then we can brainstorm, okay, uh if we have, we'll first, before we even arrive on the site, we probably want to know beforehand that we are going to do some kind of a work at height activity, for example. So then we know what kind of information to bring. Okay, well, then we can already start with that and move away all the way up to the uh end of the control where we're cleaning up and making sure that the materials are stored in the correct position, and these kind of things. So those are the operational steps. Then uh from the technical point of view, we want to identify the types of equipment and systems required for the functioning of this tool. So, what kind of tools, for example, do you need to bring? Um, what kind of technical requirements do these uh tools have? You can specify them as well. And lastly, and but not least, of course, the organizational bit, which typically focuses more on the roles and the functions and the competencies required for you to execute this. So, how many people do you have to have in order to do this? What kind of competencies do these people need to have? What kind of information needs to be already sorted out beforehand, for example, so that when people arrive to implement this control, they have all the information and uh uh and and skills, basically, required to do the job.
So let's take a look. We'll just use the fall arrest harness because I think that's quite a ubiquitous uh type of control that most people will be familiar with to some extent at least. Uh, it's of course an example, right? So we're not going into too much detail, but uh we could say that the control is the use of the fall arrest harness or the full arrest system. So we have a harness, we hook it up to something to make sure that if we fall, we don't hit the floor. And that's exactly what the the control function tries to describe. So the the use of the fall arrest harness system should prevent a person from hitting the floor or the structure after falling, for example. Maybe that's that's that's what we see that this thing needs to achieve. This already provides us with some information on how to measure this control. Like, ideally, if we measure this control, we should be able to predict if somebody, if this this device will actually prevent this person from hitting the floor or the structure, for example. But uh since this is sometimes difficult to measure straight out of the bat, we also want to look at some of the requirements through the operational, technical, and organizational lens. So from an operational point of view, we might see something like this. So it's almost like a timeline. So at some point, receive the work order, we collect our gear, before we put it on, we check our gear, then we proceed to putting it on, then we hook it onto the structure, we execute the work, and then afterwards, we check our gear and maybe return it to the uh the the the position. Since my screen was this big, I left it to these four steps, but you can imagine that in real life, you would be a little bit more minute in in describing this, but this is an an example.
Well, those are the operational steps. So that that's basically what needs to happen on the day of execution. Well, then there's also some technical parts related to it, of course. There's the harness to think about. The the harness needs to be in a good condition. But in order to retrieve the correct gear, probably the harness also probably needs to be equipped with some kind of a tag or unique identifier. So we know who is wearing which harness and keep track, for example, of the uh the certification of such a harness. But other than the harness, of course, there's also the fixation points, because if we have a perfect harness, but we don't have a point to hook it on to, there's no real point, of course. So also this needs to be from a technical point of view, needs to be in place. And then lastly, for the organizational point of view, um, yeah, there's other things we can think about. So probably want to make sure that the the gear is available in the in the correct sizes. So we need to make sure that we know, yeah, the sizes of the personnel working on that site, and they should be available and and certified. We need to make sure that the person is available on site to to do this check. So is everybody capable of doing that check? Are people trained in how to do that check? Are they trained in how to wear the harness, for example? All these competencies can can be listed here. Oh, there's a duplication here. But also when returning the gear, for example, you can imagine that you might want to have somebody there to take in the equipment, maybe do an additional check, or at least store it in the correct facilities.
So with a form like this, and typically we do this on a large whiteboard, for example, or a a big wall with Post-it notes, we try to really break apart this control and identify these different smaller requirements from the operational perspective, technical, and organizational perspective. And every one of these little green cards on the on the board are potential hooks where we can identify some uh uh some some data points on. For example, it doesn't mean that we have to to measure everything. It's just trying to, well, first of all, understand what uh what this control is all about. So what kind of activities go on underneath the hood of this control. But then also, yeah, provides us uh maybe some options of things that we can measure. So if we can't measure, for example, the function of the control directly, then maybe we can measure some of these requirements. And if we know, if we see that one of these requirements is not being met, or it tells us something, of course, on the uh uh the quality of the control at that point. Um, and it's always an eye opener, I find, like if we do this, uh, yeah, when when you're making the bow tie, it can be just very simple to put these words in the box and say, okay, we have a fall arrest system uh for this uh this risk. Everybody sort of understands what it means, so that's fine. But then when you dig a little bit deeper into it, you suddenly see, oh, there's actually quite a lot of activities and and stuff that needs to happen for this uh fall arrest harness to really work.
Uh, yeah, and then uh it can be good for that. A similar system also works with, I guess, less active types of controls. So when we're thinking of, for example, like a bund wall. So this this would be like a little dike around uh like, let's say, a tank that holds oil. If that uh, yeah, primary form of the tank, for example, fails, the oil will spill out. If we have a bund wall, it's quite a nice way of containing that spill into a particular area, so that we can can deal with it. So again, the function of that bund wall is to prevent fluid from reaching the environment after spilling out of its primary containment. It's quite an effective control, of course, like, uh, even if the the tank fails, we still are capable of containing it. But it's more of what we usually discuss by as a passive control. It doesn't require a direct human interaction, for example. It's just a wall that basically sits there. But even on these kind of things, we can still look at it from an operational, technical, and organizational point of view, although probably it will be slightly heavier on the technical aspects compared to the other ones. But yeah, probably we want to make sure that the uh that, yeah, the wall of that uh uh that bund wall is uh uh structurally integral, so that it is indeed capable of containing the fluid. But typically, because we're building a bit of a dike there, there needs to also be some kind of a drainage system, because if it rains, then we don't want all the rain to pile up in that. We eventually want to be able to spill it out to to get rid of that rainwater. So there probably should also be a drain system. But at the same time, if there is a drain system, system, we also want to be able to close that drain system off, so that if there is indeed an oil spill, that we don't drain all the oil away, uh, as we would with the rainwater. So again, uh, there's some technical capability. So of course, the wall needs to be high enough so it can contain all the fluids and stuff like that. What we notice with these kind of more passive types of controls is that, uh uh, you typically don't want to measure that every single time, because it's just a wall. It will be sitting there, probably for quite a long time, for example. There will not be any change to this wall. Uh, if you check this every day, you will get the same results, probably for a very long time. So maybe it is okay to check these kinds of controls in a slightly uh uh longer types of cycle. So again, also when when thinking about the life cycle or the uh the execution of the control, the types of control can also dictate how often we should measure it. Um, but again, yeah, it's try it's fun to to to try to decompose a control and uh uh identify what kind of aspects are in it. Uh, so we can actually uh uh, yeah, see what we can measure within the control as well.
Uh, of course, uh, I mean, if we're measuring, uh, uh, well, breaking down these, this individual activities, we can measure those activities, of course. But there is a bit of a a pitfall there. And that's basically is that we, the further you stray away from the actual barrier function, so the further we start measuring requirements, or maybe even the supporting activities that need to make up that requirement, the further we are removed from the actual controls function. So we're starting to measure things that uh are more precursors to the functioning of the control, which is sometimes uh just easier to measure. So that's why we often do that. But it also becomes less and less predictive of the controls function, the further we are removed from the actual uh Control Function. So typically, if we can measure the Control Function directly, uh, like prevent a person from hitting the floor, uh, uh, or structure after falling, then that I should ideally be our focus of measurement. Unfortunately, that's not always possible. Like, for example, with the uh the fall arrest harness, we can't go, I mean, I don't, I don't think it is a good idea to ask people to, can you please fall for me so I can check if your harness is working? That's probably not the best approach. Uh, because we're exposing people to danger. But there are types of controls where you can actually do this. So, for example, uh, if we have a lifting crane, for example, there might be some kind of a software device which uh prevents you from extending the load too far outside of the reach of the arm, making the crane collapse, for example, because the the pressure is uh become too high. Uh, you can actually test directly if this control is working. You can just take the load, lift it a couple of centimeters above the ground, and then try to extend it as far as you can, and then at some point, this uh uh control device should kick in. If it doesn't, well, we know that control has failed. So that's a good indication. Uh, and if it does work, okay, we have proved that the control is working without doing actual damage to the crane, and then we can start the operation. So, uh, if you can measure the Control Function directly, that's always the best, because then you're directly measuring it. But like I said, not all of these controls allow you to do this, because it becomes just simply dangerous to try and and actually test it real life.
Um, so in that case, it might be good to look at some of those requirements and say, okay, maybe we can measure some of these requirements, and if we know that these requirements are not there, we can also tell us something about that the controls might not be as effective as we would like. Um, so, for example, we, uh, when we're checking the the fall arrest harness, for example, we might be able to check, okay, is the harness in good condition? Uh, are people wearing it correctly? These kinds of requirements. Now, you can already see perhaps or think about that this already poses some problems. But because if we're checking the harness and the harness is okay, it's not a foolproof guarantee that people will not hit the ground. Because if they're not hooked up to a correct fixation point, or if the fixation point, for example, fails, the harness can still be in perfect shape, but it's still will not provide the uh the kind of risk reduction we're looking for. So that means if we're doing requirement checks, we probably have to check quite a few requirements in order to come to a full complete picture if we we really want that full control uh complete picture. And that becomes even worse if we are looking at supporting activities. So the supporting activities would be, yeah, all kinds of tasks we do to make sure that the requirements are being met so that in the end, the control works. Again, then we are even further removed from the actual uh Control Function. And uh then it becomes even more wonky. The thing is that usually these these supporting activities are just easier to measure, because maybe we indeed have a certification regime, and we can check the stamps, and we can see that it is there, and that makes for very easy checking. But unfortunately, it's not always super predictive for the actual Control Function.
Um, so keep that in mind. So when you're trying to measure the control, always try to think about ways, can we really check the Control Function? If not, can we check the requirements? And if if that's not the case, then we can maybe still check some of the supporting activities. But the further you are removed, the, yeah, the more careful you need to be uh about what is this data really telling me.
Uh, yeah, I hope that makes sense. Um, then a question about how to measure these kind of things. So, what kind of measuring tricks can we do to uh to come up with h with, yeah, the the barrier performance data? Well, we typically see these types of information being used, and this this is definitely not a uh complete list, I would say, but this is generally the rough areas I feel that people uh come into. And all of these methods, I think, have their merits and uh uh and downsides. So it's not like we should always choose one or the other. Uh, there are different benefits and and downsides to these, and I'd like to quickly go over them.
Uh, so we have opinion, operational. Uh, so opinion is basically just asking people, do, how do you feel this control has been working? Uh, operational data would be more about uh sensor data, for example, or maybe some other type of log that we can can assess. We have inspection data, and that's basically uh maybe a quick checklist or something like this that we can uh can do, or a quick walk around where we check certain things. Incident data, so have these controls been working in the past, or have we seen incidents where they have not been working? And the last one is audits, which I see as slightly different than the inspection. So I think audits are usually lower frequency and also looking at more of the supporting activities often, rather than the inspections would be more on the work floor directly inspecting these items. Uh, yeah, like I said, there might be more in here, but this is what I want to confine myself to. And I've created a little bit of a scorecard with a couple of characteristics and how these measurement types score on this scorecard. So the first one is opinion. And often, uh uh people don't like this because, like, it's an opinion, like, how, how, why, why they can be very uh inaccurate, right? So people can just say that they feel the controls working while actually it is not, for example. So is it useful? Well, I think it's an over uh overly uh underlooked type of control. I think I quite like this one. Um, because the effort to set it up, to set up a process to capture opinion is quite simple. Like, in a lot of organizations, we can just create a Ms form type of automated form and just send it out to people from time to time. You can ask a lot of different people how they feel this control is working. And if it's a control that people work with on a daily basis, for example, they have a very good understanding often how this control is working. And since they are working with it every day, they also have all the reason to make sure that this control should be effective and easy to work with, for example. So it's relatively low effort to set up.
The effort in operational units, what I mean by that is basically how uh uh intrusive the the measurement is to people on the work floor. And this is, yeah, it can be a bit intrusive. Like, if we, we do this every after every job, for example, people get fed up with it very quickly. But typically, these opinions can often be gathered maybe on a quarterly basis. And if you just have to spend one minute of your time every quarter to to check in on this control, can be very simple. And I think uh the uh, yeah, the the impact it will have on people's time is still relatively low. Uh, if you do it very often, of course, it gets tedious, so you have to uh uh scale that a bit. Uh, it uh also applicable to almost any control. So you can do this for anything. You can do it for bund walls, if you want to, like, or you can do it for the the harness. It usually works. Uh, the accuracy, of course, yeah, can be sometimes low, especially if people are incentivized to give off green readings, for example. But often, what I find is that the accuracy is quite uh good. So if we're asking people, like, people are genuinely often concerned with these kind of things and do actually report issues that otherwise would not have been found. Um, and specificity, it is quite nice because it captures often uh quite a broad perspective on this control. People can come up with anything that they want to say about it. They can can mention it. We're not spec uh specifically asking about a specific part of the control. We're just asking about the general control in and of itself. What often find is like, most organizations, of course, have like some kind of a system in place where people can leave a safety report if they see something that's out of the ordinary. It is often quite a big hurdle to actually do this. So often people, I think, will see things that are out of the ordinary, but yeah, don't have the time, or uh don't at that particular point in time have the time to to write it down and actually put in a whole form. Whereas if we are actively going out to people and asking their opinion about this specific control, then they will often say, well, actually last week I saw that uh these these harnesses, for example, were, it didn't fit, like I didn't have a good harness and I just had to go to work, but it wasn't comfortable, it wasn't something that I would like to do. People don't often fill out like a full form for this at that point in time. But if we ask them later on, they still bring up this kind of information. And we actually, when we do this with clients, we often find very interesting and actionable items, just through this this means, but slow effort and still have has quite good results.
Operational data is very interesting as well. So this would be, for example, sensor readings. And even for things like working at height, like the fall arrest harness, we have seen uh companies that use, for example, uh hooks or uh sensors in those hooks. So you can see actually where people uh attach it to uh uh or if it has been attached, for example. So there are sensor data sometimes that we can can get in here. I think nowadays with the Internet of Things type of applications, we see more and more sensor data coming online that we can perhaps make use of. Uh, it's typically very difficult, however, to set that up, of course. Like, have you have to install sensors everywhere? That's quite a big investment. But the the upside, of course, is that uh once the sensors are there, you don't need any human input anymore. You can just read out those sensors. So that's a that's that's a big plus. So if you do uh many frequent measurements, then this is they can be a very effective way to go. And sometimes the sensors are already there for other purposes, and then you can reuse the uh the effort. Applicability, ever is quite low, because there's only a few, yeah, a handful of controls typically that uh that really lend itself very well to this types of measurements. But it can be done. Uh, the accuracy is often pretty high. You can just get the numbers, you still need to interpret them, of course, but uh the accuracy should usually be good. Uh, but the specificity is uh that it's usually quite topical. Like the sensor is only measuring one particular thing. Like, if we have a sensor in a hook where people hook it onto, for example, yeah, that tells us that people have been using that hook, but we don't know if that hook has ever been
attached to a harness, for example. So, yeah, we're only measuring this very specific thing, which is, uh, uh, yeah, just, yeah, the the reality of these kinds of things. But it can still be useful, of course.
Um, inspections, like, uh, uh, sometimes this can be done as a sort of a pre-shift type of inspection. Like, before we start work, we do a quick round check if all the critical controls are in place, and then we we start the shift. Um, that can be an IMPC, a way of doing it. Or, for example, before we start working on high voltage equipment, we want to make sure that at least, uh, some essential steps have been done, uh, for example. So that that could be sort of a type of an inspection thing.
Um, it's a bit more effort to set up because you have to come up with a good way of of measuring this. Maybe you created some kind of an app, perhaps, to to to get the data in here. U and it's also typically quite high effort for the people on the workflow because there needs to be someone who does this check. Uh, it costs time every time. Um, but, yeah, it's can can be a very direct way of measuring, of course. The nice thing about this, of course, it's applicable to most controls. It's quite high accuracy, uh, and it can cover quite a broad range of aspects. So it can be a quite a good tool. But I think it has to be used sparingly and effectively because it is quite impactful on the people working with it.
Um, then we have, of course, incident statistics, um, which can, um, yeah, you can read the scorecard, perhaps for yourself. So it's, uh, it's quite high to, uh, to set up, I think. Like, uh, setting up a good incident analysis and doing a good analysis takes quite a lot of time. You have to have people that are skilled in doing that as well. Um, it is quite applicable. It's usually quite high accuracy as well, if you have enough time to spend on it. U but, of course, the downside is that, uh, uh, it is all, of course, very, uh, retrospective. And also the in the data coming in is very incidental. So something has to go wrong in order for us to do an incident, uh, uh, on it. And even if something went wrong, but just for for sheer luck, it doesn't didn't result in an unwanted outcome, then we're usually not capturing that kind of information just by using incident statistics. So there are some some downsides to it. But it also can lead to interesting insights.
And the last thing is the the audits, uh, uh, which is, of course, high effort to set up and relatively high, uh, for the operation, high of an impact on the operation because people will have to take. I often have like visits with clients and they say, well, there's an audit coming up and everybody is like super busy with the audit. All appointments are cancelled. So you can see that it's usually quite impactful on the operation to do this. Um, but the other, yeah, the the the upside of it is, of course, that is quite broadly applicable and you can measure all kinds of things. But this is typically not something that you would want to do obviously on a daily basis. This is very low frequency type of stuff, but still useful.
Um, so these are these kind of kind of things. So why do I bring this up? Uh, well, just to give you a bit of a showcase of what kind of measurements you can think of. Um, but also to give you an idea that, uh, you don't have to like always go for operational data or always go for the inspections. You can choose whatever fits best for the types of control. And you can also say, well, maybe we want to start off with the low effort stuff that gives us maybe a little bit of insight, like an opinion type of thing. It's pretty a straightforward setup. And then if we feel that in the opinions, we can we see that the reports are coming into the control is not working as well as we would like, uh, then we can decide to to ramp up to more effortful types of investigations. Because then you have also for the organization, like, uh, you can demonstrate, like, okay, we already see that this control, people have spoken up that this control is not working as well as we should. So perhaps it is warranted to take a little bit of a closer look, maybe by installing sensors or maybe by, uh, uh, putting in an inspection type of regime, which is a bit more intensive, perhaps. But there is a good reason for doing it because, well, uh, we see that it that is problematic. So that's why, uh, I think it's always a good idea to start with the low hanging fruit, like the opinion one, and then build your way up to more, uh, intensive, uh, regimes if needed. And for a lot of controls, it might never be needed. So we just use the opinions to to periodically check in on it and only ramp up when we actually want it to be in place. So that's good to think about as well. Don't you don't have to go all the all the way, uh, straight from the bat. You can just build your way up if if needed. And it's also easier to get the organization along with you because, well, we already have some data that says that it's not okay. So maybe we should take a look a bit of a closer look. Or for example, I often see like, uh, yeah, well, this is just opinion data. So don't we have any hard data? Well, if they already say that themselves, then it's easier to go into that direction.
Question. Well, then the last question, of course, uh, should we measure all these things? Because any time of measurement, of course, costs time. Do we want to, uh, uh, to do this? Um, and I think a practical way of looking at it is, uh, uh, through this sort of a matrix, uh, where there is a sort of a a need for the data. So like how how how badly do people want to know about the quality of this control, which can be a low, medium, or high. For example, if the if the people already feel that the control is probably working fine, we haven't seen any incidents with this, then probably the need to check this control is relatively low because people already feel that this control is quite is working quite well. Whereas there might also be controls that, uh, are known problem areas or people are really queasy about this. They don't they have seen incidents with it, for example, or, uh, it is a new type of, uh, thing that they're doing and they're just not sure about it. Then the information need is quite high because people want to know how well this control is doing.
Um, and then we can set it off against another AIS about the amount of effort that is required to obtain the data. So for some D for for some types of controls, the sensor data is already available. So there's very low effort to really get the data because we already know. Well, in that case, like going for active measurements is quite an easy step to take because it's, uh, it's very little effort. We can just get the data straight away. But there might also be, for example, um, the, uh, uh, the, uh, very high, uh, uh, effort types of data to collect. So, for example, there's it's just a very difficult control to measure actively. For example, yeah, we almost have to go out into the field with the clipboard to really start counting these kind of things. Super high effort. Um, that's probably not something that you we want to do for the the the if the data need is low, then and the the the amount of effort we have to put in to get the data, probably the answer should be no. Like we don't want to measure these controls in that case because it's just too much, too impactful for the situation.
Um, then there's, of course, also a bit of a gray area. And I my advice usually is to to start with those periodic types of checks, uh, in these kinds of cases. So, um, then you can still like because it's periodic, like maybe every quarter or, uh, uh, on a monthly basis, depending on what you want to do, uh, you can still get some information on it. And based on that, you can then later decide to go into a more active measurements or more frequent, uh, measurements, or maybe decide not to measure it any further, or maybe reduce the frequency even further. So maybe a check once a year, for example. Um, that still, yeah, keeps the the measurement, uh, load manageable, I think. Uh, so you don't have to always go for active measurements. You can just play around with the frequency and find unit the way you think is, uh, is best.
Um, yeah, so, um, some some final conclusions and takeaways. I think we're almost at the end as well. Um, so depending on your goal, uh, infrequent measurements can still be valuable. So sometimes people think, okay, we're doing critical control management, which means that every time we do an operation, the control needs to be fully perfectly in place, otherwise we don't start. That's of course one of the goals that you can set for yourself. But even if you, uh, if you measure more infrequently, you can still improve the control over time, and it can still that can be an acceptable first step. And along that way, uh, and I think that that should be, uh, uh, U, um, that's that's commendable as well, I would say.
Um, try to measure the control as directly as possible. So what I mean there is to try to measure the Control Function ideally directly, if you can, and then move away from it if if that's impossible. But like then look at the requirements and then the supporting activities. Um, you want to have a good understanding of what the control is before you start measuring. So trying to make such a control breakdown, for example, in the manner I I shared with you today.
Um, and then last one, uh, take a pragmatic approach. So start measuring it in easy ways. So maybe just an opinion type of poll, and then you can always make it more complex later on and more frequent, for example, with, uh, more interesting, uh, measures. Um, but, uh, to start off with opinion is not a bad idea. And then you can just find chewing in it as we go along. And then, of course, not everything has to be measured. So if it's super difficult to get the data and the control is just not that exciting anyway, then probably you don't want to measure it in the first place. And that's also okay to to draw the conclusion.
Um, yeah, that's as far as I wanted to take you, uh, today. Lot of information. I I bet. I hope it was easy easy enough to follow. And, um, yeah, if there if there's any questions, I'd be happy to, uh, to take them. No questions forthcoming. Ah, yeah. Ah, Hello Evok. Nice to to see you. Uh, so the question is, uh, from EV is, what methods or strategies do you typically use to report, summarize results, and measure, uh, on critical controls? Uh, good question. So, uh, since the topic of this, uh, uh, uh, presentation was mainly on the measurement part of it, um, we haven't really covered this evaluation part, which is basically like how do we present the the the resulting things.
Um, it depends a bit, I think, like how do you want to present the data that is incoming depends a bit on how frequently you're doing it. So if it's like a quarterly check with opinions, for example, then just making a PowerPoint presentation every quarter with the results might be perfectly fine. If you're doing it more frequently, like, uh, uh, on a monthly basis or even more often, then maybe a dashboard becomes more suitable. So you can actually see, uh, trends over time, for example. So it already controls improving, uh, going down. Uh, I think that's usually best done in a sort of a dashboard environment because the the frequency of data updating is, uh, is more logical. And, uh, yeah, depending on the client, we've we've done that, for example, in Power BI, where you have a U generic boarding type of platform, or Tableau, or something like that.
Um, but, uh, yeah, like I said, in other ways, you can also just simply make a presentation. That can also work. But if you have to do that every week, for example, then it very gets old very quickly. So you want to have a more, uh, direct measurement of that, or like a direct way of displaying it in a dashboard. Does that I hope that answers your question to some extent? And nice to of you to to join there. Okay. Any further questions? Otherwise, I'd like to thank you very much for for attending. I hope you enjoyed it. Uh, yeah, the recording will be published, uh, later, uh, maybe today. I don't know, like at some point. And feel free to share it, of course, with your colleagues. Uh, thank you for your attention today, and, uh, yeah, I hope to see you with the next one. Thank you very much. ES thanks you. Bye-bye.