D ELFT U NIVERSITY O F T ECHNOLOGY M ASTER T HESIS S YSTEM E NGINEERING , P OLICY A NALYSIS AND M ANAGEMENT MEASURING THE IMPACT OF INFORMATION TECHNOLOGY SYSTEMS ON POLICE PRACTICES IN THE DUTCH POLICE FORCE D EVELOPING A MEASUREMENT TOOL FROM A SYSTEMS THINKING PERSPECTIVE Author: Davy Osamulia Student ID: 1309862 Graduation Committee: Prof. dr. ir. Alexander Verbraeck Dr. ir. Mariëlle den Hengst Dr. ir. Mark de Reuver External supervisor: Drs. ing. Ruud Hoefnagel th
October 31 , 2014 Measuring the impact of information technology systems on police practices 1 2 Measuring the impact of information technology systems on police practices PREFACE & ACKNOWLEDGEMENTS This master thesis is the final part of a two-­‐year Master’s Programme in Systems Engineering, Policy Analysis and Management. This thesis presents the results of the research project I conducted at the Dutch Police force. This master thesis also marks the end of my study period. During this period I gained interest in the complex, synergistic relationship that technology and society has. Being able to spend my master thesis on such a topic at the Dutch police force has therefore been a great opportunity. At the police force I was able to learn and experience about many aspects involved with technological change at such a complex organization. Having had this opportunity, I sincerely hope that the results of this research will have a contribution or at least inspire those concerned towards better understanding and mapping of technological impact. The results of this study could not have been achieved without the support of many people. Therefore, I gratefully acknowledge the graduation committee. I would like to thank Mark de Reuver for his time and effort as a second supervisor and Alexander Verbrack for his flexibility standing in for Ms. Brezier as a first professor. I would like to thank Marielle den Hengst for introducing me to this topic, and also for establishing contacts with Ruud Hoefnagel. Marielle den Hengst has always been enormously helpful as a first supervisor. She always had time to discuss and brainstorm about struggles that I had, especially during the beginning when formulating the research topic. I am equally as thankful for Ruud Hoefnagel, my practical supervisor. Ruud Hoefnagel was always enthusiastic and available for help, even in his spare time. His abundance of knowledge and expertise as a member of the police organization and as a researcher has also helped me tremendously. I would also like to thank all the operative colleagues that helped me conduct interviews and card sorting sessions. It was amazing to witness how people from the police force were always willing to help. From administrative workers all the way to high commanding chiefs, they always managed to reserve some time for me. I am especially thankful for the regions Den Haag and Rotterdam who have gone out of bounds (figuratively) in helping me. Thank you BETEX (Den Haag) for letting me follow you guys around. Thank you Rotterdam for showing me how you guys get things done. I would like to thank Marijn de klein, my fellow intern buddy with whom I had many interesting brainstorms and discussions about our research experiences and future paths. Last but certainly not least; I would like to thank my parents and girlfriend for the support they gave me. Measuring the impact of information technology systems on police practices 3 4 Measuring the impact of information technology systems on police practices EXECUTIVE SUMMARY (NL) Aanleiding Uit een rapport van de algemene rekenkamer gepubliceerd op 23 juni 2011 is gebleken dat IT voorzieningen van de politie het politiewerk onvoldoende ondersteunen. Naar aanleiding van dit rapport is er binnen de politie het aanvalsplan informatievoorzieningen (AVP) opgezet. De AVP moet door middel van een investering van 365 miljoen euro aan modernisering van huidige IT voorzieningen zorgen voor verbeteringen in de mate waarin de IT voorzieningen het politiewerk ondersteunen. Om de voortgang van deze verbeteringen te monitoren is daarvoor de belevingsmonitor in het leven geroepen. Voor de opzet van de belevingsmonitor is gekozen om indicatoren op te nemen gebaseerd op een technologie acceptatie model genaamd UTAUT. In hoeverre het huidige ontwerp van de belevingsmonitor voldoende inzicht biedt in hoe politiewerk verbeterd wordt door IT, is een vraag die leidend is in de uitvoering van dit onderzoek. Methodologie Voor er aanbevelingen naar de belevingsmonitor gemaakt kunnen worden is het van belang om te weten wat technologisch impact op politie werk eigenlijk is. Politiewerk, in tegenstelling tot productiewerk, is complex en niet te reductioneren tot een aantal elementen die een lineair proces belopen. Bovendien is de impact die informatie technologie op politiewerk heeft ook niet te isoleren tot een aantal selectieve elementen, informatie technologie heeft impact op het geheel. Door de complexiteit van wat technologisch impact op politiewerk is, hecht dit onderzoek belang bij om technologisch impact op politiewerk in zijn geheel te kunnen zien. De focus op een holistisch system thinking aanpak was daarmee ook leidend in dit onderzoek. Met het system thinking aanpak werd getracht om het gehele politiewerk systeem dat beïnvloedt wordt door informatie technologie zo compleet mogelijk in kaart te brengen. Dit werd gedaan door een zogenaamd mental map te maken van het systeem, dat een visualisatie betreft van de concepten waaruit het systeem bestaat en de relaties ertussen. Door eerst een mental map van het systeem als een geheel te zien, kan van daaruit verder de meest relevante indicatoren uit worden gekozen. Voor de constructie van de mental map van technologisch impact op politiewerk zijn er meeloopdagen en interviews gehouden met tien verschillende eindgebruikers die dichtbij het politiewerk zitten. Dit zijn niet alleen operatieve politieagenten, maar ook administratieve medewerkers en leidinggevende/coördinerende politieagenten. Op basis van deze interviews zijn er 42 relevante concepten geïdentificeerd die betrekking hebben tot technologisch impact en politiewerk. Voor de selectie van de meest relevante indicatoren voor meting zijn deze 42 concepten opnieuw voorgelegd bij een groter groep van 25 eindgebruikers waarbij gevraagd werd om deze 42 concepten te sorteren op basis van relatieve relevantie. De sorteersessies resulteerde in relevantiescores voor elk van de 42 concepten. Op basis van deze kwalitatieve en kwantitatieve empirische data maakt dit rapport aanbevelingen naar de belevingsmonitor over welke indicatoren meer of minder geschikt zijn voor opname in de belevingsmonitor, met inachtneming van het doel om inzicht te krijgen in de impact van technologie op politiewerk. Daarnaast wordt er in dit rapport ook aanbevelingen gemaakt hoe meting van deze indicatoren gerealiseerd kan worden in de belevingsmonitor. De aanbevelingen worden gemaakt vanuit een contingency approach,wat betekent dat er niet geen goede of foute manier van meten is. Vanuit deze contingency beredenering stelt dit onderzoek daarom niet een mogelijke verbetering, Measuring the impact of information technology systems on police practices 5 maar schetst het een aantal toekomstige meet scenario’s voor om de eigenaar van de belevingsmonitor te assisteren in zijn overwegingen voor mogelijke aanpassingen. Resultaten Op basis van interviews zijn er in dit onderzoek maar liefst 42 onderwerpen geïdentificeerd die relevant zijn voor monitoring op het gebied van technologisch impact en politiewerk. Voor de executive summary worden de 15 belangrijkste indicatoren toegelicht. De 15 belangrijkste indicatoren komen voort uit de relevantiescores die gegeven zijn in de sorteersessies. Per indicator wordt een omschrijving gegeven wat de indicator is en een generieke omschrijving omtrent de motivatie voor de relevantie van desbetreffend indicator voor monitoring. 1.
2.
3.
4.
5.
6.
7.
8.
9.
6 Uitvoer van accurate informatie. IT moet ervoor zorgen dat de informatie die eindgebruikers krijgen accuraat is en zonder fouten. Op dit moment bevat de informatie verkregen uit de huidige systemen nog teveel fouten waardoor dit onderwerp gerankt is als meest relevant. Tijdsgebruik. Het gebruik van IT vergt op dit moment nog teveel tijd en moet in de toekomst zo min mogelijk de tijd van de eindgebruiker belasten. Met name voor operatieve politieagenten op het veld is minimale tijdsgebruik van IT voorzieningen cruciaal. Uitvoer van complete informatie. Naast het feit dat informatie verkregen uit IT voorzieningen accuraat en zonder fouten moet zijn, is het ook van belang dat de verkregen informatie compleet is. In gevallen waar verkregen informatie correct en zonder fouten is, kan het ontbreken van informatie alsnog leiden tot onopgeloste zaken of gevaarlijke situaties voor agenten op straat. Invoer van accurate informatie. Op dit moment werken huidige IT voorzieningen haast ontmoedigend in het ondersteunen van agenten bij het invoeren van accurate informatie. Toekomstige IT voorzieningen moeten ervoor zorgen dat eindgebruikers zoveel mogelijk ondersteund worden om de juiste informatie in te voeren. Gebruiksgemak. IT voorzieningen zijn nog steeds niet gebruiksvriendelijk genoeg. Er zijn teveel handelingen nodig om gegevens op te vragen of in te voeren. Dit gaat uiteindelijk hand in hand gaat met tijdsgebruik. Uitvoer van tijdige informatie. IT voorzieningen faciliteren onvoldoende in de tijdige uitvoer van informatie wanneer het nodig is. Vooral op straat is tijdige uitvoer van informatie cruciaal. Wanneer een agent een individu aanhoudt is het van belang dat een agent gelijk de juiste informatie voor handen heeft. Administratief werk. Agenten ervaren dat IT implementaties gepaard gaat met meer administratieve lasten. IT moet ervoor zorgen dat de administratieve last van eindgebruikers, en voornamelijk van operatieve agenten verminderd worden waardoor agenten meer tijd op straat voor operatieve taken door kan brengen. Kennis behoud. Door implementatie van IT worden vaak handmatige processen geautomatiseerd, wat vaak gepaard gaat met bezuiniging op administratief personeel. Het bezuinigen op administratief personeel heeft echter ook het gevolg dat daarmee het stukje specialistische kennis verdwijnt. Met oog op toekomstige IT is het van belang dat het stukje specialistische kennis behouden blijft of in ieder geval gecompenseerd wordt. Communicatie. Met de implementatie van IT wordt de afstand tussen gebruikers groter en communicatie soms onpersoonlijker. Het is van belang dat toekomstige IT implementaties genoeg mogelijkheden biedt om communicatie tussen eindgebruikers te blijven ondersteunen en te waarborgen. Measuring the impact of information technology systems on police practices In het onderzoek is ook geconstateerd dat er significante verschillen waren tussen indicatoren die belangrijk geacht worden geacht door leidinggevende eindgebruikers en overige eindgebruikers. Die relevantste indicatoren voor leidinggevende met het grootste scoreverschil tussen leidinggevende en overige eindgebruikers zijn als volgt. •
•
•
•
•
•
Taakinnovatie. Leidinggevenden hechten waarde aan het feit dat IT hen niet in hun bewegingsvrijheid belemmeren en hen mogelijkheden bieden om bepaalde werkzaamheden uit te voeren op wellicht nieuwe alternatieve manieren die volgens hen beter geschikt zijn. Management en controle. Leidinggevende hechten waarde aan de mogelijkheden die IT voorzieningen bieden in het aansturen en controleren van hun team. Op dit moment zijn de mogelijkheden op dit gebied nihil en biedt IT onvoldoende ondersteuning in dit aspect. Acceptatie. Leidinggevende hechten waarde aan de mate waarin hun team nieuwe IT accepteren. Acceptatie van IT betekent dat teamleden uit zichzelf bereid zijn om IT te gebruiken, wat zou kunnen leiden tot minder noodzaak in controle en aansturen tot gebruik. Productiviteit. Leidinggevenden zijn direct verantwoordelijk voor productie. Leidinggevenden zullen daarom ook veel waarde hechten wanneer IT ervoor zorgt dat productiecijfers zoals opgespoorde individuen, betaalde boetes positief toenemen. Efficiency. Naast het behalen van productiecijfers is het ook van belang dat deze cijfers op een zo efficiënt mogelijke wijze gebeurt, met minimale belasting op personeel en andere schaarse middelen. Pro activiteit. Leidinggevende stellen het op prijs dat IT mogelijkheden biedt voor agenten om proactief te anticiperen op nieuwe informatie. Pro activiteit van teamleden betekent namelijk dat leidinggevenden minder exacte instructies hoeven te geven of om agenten telkens opnieuw op pad moeten sturen omdat zij bepaalde alternatieve oplossingen niet zelf hebben geprobeerd. Aanbevelingen Op basis van de inzichten in dit onderzoek, worden er vier meetscenarios geschetst met betrekking tot toekomstige continuering van de belevingsmonitor. De eerste meetscenario is het continueren van de belevingsmonitor in de huidige staat terwijl de andere drie meetscenarios per opvolgend meetscenario incrementeel meer aanbevelingen bevatten en dus meer aanpassingen aan de belevingsmonitor vergen. Per meetscenario wordt besproken wat de implicaties zijn voor besluitvorming is voor het kiezen van een bepaald meetscenario. •
Scenario 1: huidige staat van de belevingsmonitor De huidige opzet van de belevingsmonitor, bestaande uit 19 enquêtevragen gebaseerd op UTAUT die online gedistribueerd wordt over het gehele land, wordt voortgezet. Het voordeel hiervan is dat er geen additionele middelen en geld ingezet dienen te worden voor de uitvoering van de belevingsmonitor naast de middelen die nu al nodig zijn. Daarnaast is vergelijken van scores op indicatoren tussen nieuwe en oude uitgevoerde metingen met de belevingsmonitor daardoor nog steeds mogelijk aangezien de indicatoren gelijk blijven. Het continueren van de huidige staat van de belevingsmonitor heeft echter ook een belangrijk nadeel voor besluitvorming. Uit dit onderzoek blijkt namelijk dat de indicatoren die nu in de belevingsmonitor opgenomen zijn, onvoldoende representatief zijn voor de indicatoren die het relevantst bevonden worden door politiemensen zelf. Voor besluitvorming betekent dit dat het beeld op impact van informatie technologie op politie mensen ook onvoldoende in Measuring the impact of information technology systems on police practices 7 •
•
•
8 kaart gebracht wordt wanneer de huidige staat van de belevingsmonitor ongewijzigd blijft. Daarom beveelt dit onderzoek om scenario 1 niet door te zetten en om aanbevelingen in scenario 2 tot 4 in overweging te nemen. Scenario 2: een aangepast belevingsmonitor Een aanpassing wordt doorgevoerd in de belevingsmonitor, gericht op een wijziging in de enquête items in de belevingsmonitor. Uit dit onderzoek is gebleken dat tijdsgebruik, gebruiksgemak en concepten gerelateerd aan de kwaliteit van informatie ontvangst en invoering als meest relevante concepten bevonden worden door politiemensen. Aangezien gebruiksgemak al opgenomen is in de belevingsmonitor wordt aanbevolen om enquêtevragen voor tijdsgebruik, gebruiksgemak, uitvoer accurate informatie, uitvoer complete informatie, uitvoer tijdige informatie en invoer accurate informatie op te nemen in de belevingsmonitor. Om de toename in omvang van de enquête te beperken wordt verder aanbevolen om enquêtevragen omtrent facilitaire condities en sociale invloed te verwijderen. Voor besluitvorming heeft dit als gevolg dat inzichten die in kaart gebracht worden door belevingsmonitor dichter aansluit op onderwerpen die politiemensen het relevantst achten. Een ander voordeel is dat in deze scenario er minimale wijzigingen nodig zijn aan de belevingsmonitor. Er hoeven alleen enquêtevragen gewijzigd worden terwijl de data collectie methode hetzelfde blijft. Scenario 3: een aangepast belevingsmonitor + bestaande data uit systemen en rapporten Naast de aanpassingen als aanbevolen in de tweede scenario, worden in de derde scenario ook indicatoren toegevoegd waarvan data verkregen kan worden uit systemen en rapporten. Met oog op digitalisering van de processen door de introductie van nieuwe IT voorzieningen komt er een hoop nieuwe data beschikbaar. Daarnaast zijn er parallel aan de belevingsmonitor ook andere periodieke onderzoeken die uitgevoerd worden waarvan de data ook ingevoerd kan worden in de belevingsmonitor. Daarom wordt aanbevolen om de volgende indicatoren toe te voegen die zeer relevant bevonden werden door politie mensen en waarvan de data uit systemen en rapporten gehaald kan worden: o productie (doorlooptijd executietijd, executieopdrachten voltooid/maand) en administratief werk (verhouding administratief werk vs. werk op het veld) toe te voegen aan de belevingsmonitor. De data voor deze twee indicatoren zijn namelijk al beschikbaar uit rapporten van parallelle onderzoeken. o gebruiksgemak (aantal stappen nodig om informatie te extraheren), tijdsgebruik (tijd gespendeerd op systeem), output van accurate informatie (gerapporteerde fouten), input van accurate informatie (gerapporteerde fouten), output van complete informatie (aantal velden) en output van tijdig informatie (timestamps van data) verkregen worden uit systeemrecords. Aangezien de data voor deze indicatoren reeds beschikbaar zijn of al een keer verzameld is, zorgt het toevoegen van deze indicatoren ervoor dat er een breder en completer beeld verkregen kan worden van de impact van informatie technologie op politiewerk zonder dat er meer overlast is op politiemensen voor data collectie. Een nadeel hiervan is dat het voor de uitvoerder van de belevingsmonitor aanzienlijk meer moeite kost om deze aanpassingen in te voeren. De aanzienlijke toename in moeite uit zich voor de uitvoerder in het programmeren van querries in systemen voor automatische opname van data tot het periodiek evalueren en extraheren van informatie uit rapporten. Scenario 4: een gebalanceerd scorekaart In deze scenario wordt de belevingsmonitor niet meer als uitgangspunt gebruikt en wordt een geheel nieuw meetinstrument geïntroduceerd gebaseerd op het ‘balanced scorecard’ ontwerp. Het balanced scorecard ontwerp kent daarbij vijf kern themas die ‘verplicht’ zijn, waarbij de opdrachtgever afhankelijk van beschikbare bronnen en ander overwegingen een Measuring the impact of information technology systems on police practices aantal indicatoren onder desbetreffend thema kan plaatsen. Dit onderzoek beveelt aan om de volgende vijf kern themas te gebruiken: voorwaardelijke indicatoren, input van informatie, output van informatie, toepassing van informatie en de uitkomst door informatiegebruik. In figuur 0.1-­‐1 wordt een voorbeeld gegeven van een mogelijk balanced scorecard ontwerp, met inbegrip van de 15 concepten die als meest belangrijk bevonden werden door politiemensen. Aangezien in dit onderzoek 42 concepten geïdentificeerd zijn, ligt de beslissing voor het ontwerp van de balanced scorecard bij de eigenaar van de belevingsmonitor. Er kunnen indicatoren worden toegevoegd of verwijderd, maar er wordt aangeraden om in ieder geval minstens één indicator per kern thema mee te nemen in het ontwerp. Afhankelijk van de beschikbare bronnen kan de eigenaar de methode van data collectie voor elk indicator ook zelf bepalen. Er wordt wel aangeraden om een mix van subjectief verkregen data ( surveys) te mixen met objectieve data (systeem data, records) voornamelijk om de belasting van data verzameling op politie mensen zo veel mogelijk te beperken. Een voorbeeld van mogelijke data verzameling is gegeven in figuur 0.1-­‐2. Alhoewel dit meetscenario het meest afwijkt van de belevingsmonitor en daardoor de meest ingrijpende veranderingen nodig heeft sluit dit meetmodel het meest aan op de resultaten van dit onderzoek. Het is aan de eigenaar van de belevingsmonitor om te besluiten in hoeverre er zulke ingrijpende veranderingen gemaakt kunnen worden aan de belevingsmonitor. Het opzetten van dit nieuw meetmodel betekent namelijk ook dat vergelijkingen tussen oude en nieuwe belevingsmonitor onderzoeken zeer beperkt mogelijk is. Voorwaardelijk
Input
Gebruiks
gemak
Tijdsgebr
uik
Acceptati
e
Output
Complete
informatie
Accurate
informatie
Accurate
informatie
Tijdige
informatie
Toepassing
Communi
catie
Taak
innovatie
Adminstr
atieve
last
Kennisbe
houd
Uitkomst
Proactivit
eit
Productiv
iteit
Managem
ent en
controle
Efficienti
e
Figure 1.1-­‐1:Aanbevolen balanced scorecard ontwerp om de impact van informatie technologie op politiewerk in kaart te brengen (groen: indicatoren waarvan er grote verschillen zitten in de relevantiewaardering tussen leidinggevende en overige eindgebruikers) Measuring the impact of information technology systems on police practices 9 Indicator Productivitei
t
Efficientie
Gebruiksge
mak
Tijdsgebruik
Adminstratie
ve last
Proactiviteit
Taak
innovatie
Management
en controle Communicati
e
Acceptatie
Kennisbehou
d
Accurate info
input
Accurate
output info
Complete
info output Tijdige
output info
Meeteenheid Gemiddelde hoeveelheid executie opdrachten voltooid per tijdseenheid Gemiddelde doorlooptijd van een compleet executieopdracht cyclus Manuren nodig voor executie gerelateerde opdrachten Meetmethode Performance rapport Survey item:” Hoeveel tijd breng jij door achter je pc om executie gerelateerde taken te voltooien?” Hoeveelheid stappen nodig om tot compleet overzicht van een individuele executietaak te komen Tijd gespendeerd achter systeem om informatie in en/of uit te voeren Schatting vragen in enquête Gemiddelde verhouding tijd besteed aan administratieve werk/ operatief werk voor executietaken Survey item: “De mate waarin IT mij ondersteund om proactief politiewerk te verrichten” Survey item: “De mate waarin IT mij ondersteunt om nieuwe ideeën uit te proberen of taken op een andere betere manier uit te voeren” Survey item “De mate waarin IT mij ondersteunt in het managen en controleren van mijn eigen taken en die van mijn team” Survey item “De mate waarin IT mij ondersteunt in het communiceren met andere politiemensen” Survey item: “De mate waarin IT geaccepteerd is binnen mijn team” Performance rapport Schatting vragen in enquête Survey item: “De mate waarin IT ondersteuning biedt in het preserveren of verbeteren van behoud van impliciet en expliciete politie kennis en expertise” Gerapporteerde fouten Survey ratings Gerapporteerde fouten Random sampling/ Record Gemiddelde hoeveelheid velden die weergegeven worden Random sampling/ Record Gemiddeld leeftijd of information (vergeleken met datum van raadpleging) Random sampling/ Record Performance rapport Systeem record Observatie Survey ratings Survey ratings Survey ratings Survey ratings Survey ratings Random sampling/ Record Figure 1.1-­‐2:Voorbeeld van data verzameling methode voor elk van de indicatoren in de balanced scorecard ontwerp (de specifieke voorbeelden ingericht op de context van OPP E&S en zouden voor andere IT-­‐contexten enigszins veranderd moeten worden) 10 Measuring the impact of information technology systems on police practices TABLE OF CONTENTS Preface & acknowledgements ................................................................................. 3 Executive summary (NL) ......................................................................................... 4 Table of Contents ................................................................................................. 11 1. Introduction ................................................................................................... 13 1.1 Background ...................................................................................................................... 13 1.2 Research problem ............................................................................................................. 13 1.3 Research objective ............................................................................................................ 15 1.4 Research Questions .......................................................................................................... 16 1.5 Research design ................................................................................................................ 16 1.5.1 Research object ................................................................................................................ 17 1.5.2 Research perspective ....................................................................................................... 17 1.5.3 Research outcome ........................................................................................................... 17 1.6 Thesis outline ................................................................................................................... 17 2. Domain .......................................................................................................... 19 2.1 Information technology .................................................................................................... 19 2.2 Technological impact ........................................................................................................ 20 2.3 Complexity of impact on police practices .......................................................................... 21 2.3.1 Complexity of police practices being impacted ............................................................... 21 2.3.2 Complexity of technological change ................................................................................ 22 2.4 Systems thinking ............................................................................................................... 23 2.4.1 Origins of systems thinking .............................................................................................. 23 2.4.2 Systems ............................................................................................................................ 24 2.4.3 Mental models ................................................................................................................. 24 2.5 Application of systems thinking approach ......................................................................... 25 2.5.1 Motivation for systems thinking ...................................................................................... 25 2.5.2 Systems thinking framework ............................................................................................ 27 2.6 Chapter conclusion ........................................................................................................... 27 3. System Design ................................................................................................ 29 3.1 Starting point in literature ................................................................................................ 29 3.2 Design process .................................................................................................................. 29 3.2.1 Step 1: Interviews ............................................................................................................ 30 3.2.2 Step 2: Establishing interviewees ..................................................................................... 32 3.2.3 Step 3-­‐4: Identifying concepts and mental map design ................................................... 32 3.2.4 Step 5-­‐6: Verification and validation ................................................................................ 33 3.2.5 Step 7: Identification of most relevant indicators ............................................................ 34 3.2.6 Step 8: Proposing future measurement scenarios ........................................................... 35 3.3 Chapter conclusion ........................................................................................................... 35 4. Mental map of the police practices system .................................................... 36 4.1 Identification of concepts from empirical data .................................................................. 36 4.1.1 Findings from interviews .................................................................................................. 36 4.1.2 Identified concepts .......................................................................................................... 46 Measuring the impact of information technology systems on police practices 11 4.2 Mental map ...................................................................................................................... 47 4.2.1 Findings from interviews .................................................................................................. 48 4.2.2 Conceptual mental model ................................................................................................ 49 4.3 Chapter conclusion ........................................................................................................... 50 5. Validation ...................................................................................................... 51 5.1 Data collection for validation ............................................................................................ 51 5.1.1 Sampling ........................................................................................................................... 51 5.1.2 Sorting application ........................................................................................................... 52 5.2 Validation of identified concepts ...................................................................................... 53 5.3 Concepts considered to be of low relevance ..................................................................... 54 5.4 Concepts considered to be of high relevance .................................................................... 55 5.5 Validation of mental map ................................................................................................. 57 5.6 Validation of dual perspective .......................................................................................... 59 5.7 Validation of assumptions to choose systems thinking approach ...................................... 60 5.8 Conclusion ........................................................................................................................ 61 6. Designing a measurement tool ....................................................................... 62 6.1 The belevingsmonitor ....................................................................................................... 62 6.2 Reflecting the goal of the belevingsmonitor ...................................................................... 63 6.2.1 Approach .......................................................................................................................... 64 6.3 Considerations and recommendations .............................................................................. 64 6.3.1 indicators to include in the belevingsmonitor ................................................................. 64 6.3.2 data collection in the belevingsmonitor .......................................................................... 68 6.3.3 Validity of conducting before and after measurement in the belevingsmonitor ............ 73 6.3.4 Future scenarios for measurement .................................................................................. 74 6.4 Conclusion ........................................................................................................................ 79 7. Conclusion ..................................................................................................... 81 7.1 Answering the research questions .................................................................................... 81 7.2 Discussion of findings ....................................................................................................... 82 7.3 Limitations and reccomendations for further research ...................................................... 84 7.4 Reflection ......................................................................................................................... 85 References ............................................................................................................ 86 Appendix A: Interview statements ........................................................................ 92 Appendix B: Verification checklist ....................................................................... 100 Appendix C: Sorting cards used in sorting sessions .............................................. 101 Appendix D: Differences between operative officers and supervisors ................. 102 Appendix E: Distributions of card sorting data .................................................... 104 Appendix F: Literature review ............................................................................. 106 Appendix G: Categorization of levels of IS-­‐impact analysis .................................. 111 Appendix H: Design of weighted scorecard measurement tool ............................ 112 Appendix I: Interviews ........................................................................................ 115 12 Measuring the impact of information technology systems on police practices 1. INTRODUCTION 1.1 BACKGROUND With the merger of 26 police departments into a single unified national police force, the police force also intents to unify supporting ICT of the police force (Ministerie V&J, 2011). One of the stepping-­‐
stones towards a unified ICT is the development of ‘Operationeel Politie Platform’ (OPP): a project to replace all separate information systems that support operational tasks into a single nationwide information system (IS). Features that are central at the development of OPP are the unification of databases, a service oriented architecture (SOA) and mobility. •
•
•
The unification of databases is the transition from the eighteen currently separate databases into one single national database. SOA is the development of architecture with a focus on functionality. For end-­‐users, SOA will result in separate modules of OPP tailored specifically to the needs of different types of end-­‐
users. End-­‐users no longer need to use multiple IS in order to access different types of information (Ministerie V&J, 2011). Each end-­‐user will have access to a module of OPP for access to all types of information they need. The focus on mobility entails development of OPP modules that are multiplatform, thus accessible on a broad range of devices through web applications. In addition to developing and implementing new IS such as OPP, the police force is interested in how the introduction of such new IS impacts their end-­‐users. Gaining insight in the impact of IS on end-­‐
users can be valuable as feedback for further improvement of such systems, but also to provide accountability towards the minister. Currently, the police force utilizes the ‘Belevingsmonitor Informatievoorzieningen’ as an instrument to measure the impact of IS on end-­‐users. This instrument is an online survey issued nationally before and after implementations of IS. The survey items are based on the Unified Theory of Acceptance and Use of Technology (UTAUT) provided by Venkatesh, Morris, Davis & Davis (2003). Although the use of the Belevingsmonitor has provided the police with valuable insights on impact of IS on end-­‐users, the use of the Belevingsmonitor in its current form is not a given. The police force is always appreciative upon input in improving their impact measurement methods. At this point, input in improving the Belevingsmonitor is valuable since OPP is yet to be implemented, meaning that the police force can still include improvements on the Belevingsmonitor before measuring the impact of OPP. The police force is interested in how some of the new features of OPP, such as unification, service orientation and mobility introduce new elements to be measured. The appreciation of the police force combined with the possibility to improve Belevingsmonitor before the measurement of OPP forms the motivation to perform this research. 1.2 RESEARCH PROBLEM In 2008, West European countries have invested a total amount of 41 billion euros in information technology (IDC, 2013). For 2013, this amount of investments in information technology has further increased to an amount of 50 billion euros. In this fast-­‐paced era of rapid technological advances, Measuring the impact of information technology systems on police practices 13 considerable amounts of money are being spent on technology in almost all organizational structures across the globe (Yalcinkaya, 2007). Among those large organizations are law enforcement agencies such as the police force. With the launch of Aanvalsprogramma Voorzieningen Politie (AVP), the Dutch police force is reserving an amount of 325 million euros in modernizing their information technology (Ministerie V&J, 2011). Why do we invest such major amounts of money in information technology? There is a general belief that information technology eases and contributes to many aspects of our life (Nunn & Quinet, 2002). Society is bombarded with the idea of technological success (Nunn & Quinet, 2002). The idea of technological success in law enforcement is even depicted by entertainment media with the imagery of Robocop and CSI. But to what extent general belief and fiction extends to practice is still topic of popular debate. Different perspectives exist from the available literature on the extent to which information technology has changed police practices. Colvin and Goh (2005) assert that “information technology has substantial impact on police practices”. Yalcinkaya (2007) noted that “Police use information technology tools at almost every stage, including allocation of sources, patrolling, crime prevention, crime tracking, hot pursuits, and crime solving”. Maltz, Freidman and Gordon (1991) go further by stating that “information is the lifeblood of the police”. In contrast, Manning (1992) noted that “new technologies have less effect on police practices than their proponents predict or prefer” Manning (1992) pointed out, the mere availability and accessibility of information does not necessarily mean that information is used effectively or appropriately by police officers and managers. Chan, Brereton, Legosz, & Doran (2001) notes that while information technology may have enabled police to do some existing tasks better — such as recording and following-­‐up on offence details, enforcing warrants and running criminal history checks — it has not yet led to major changes in how the police deals with crime and disorder issues. Generally speaking, according to Karr-­‐Wisniewski and Lu (2010) information technology can actually be counterproductive. As American nobel laurate Robert Solow would put it, “you can see computers everywhere but in the productivity statistics” (Solow, 1987). This popular statement of disbelief in information technology actually impacting output became known as the Productivity Paradox (or the Solow Paradox). Despite the popularity of the Productivity Paradox and Solows statement, starting mid 1990s this disbelief in technological impact has shifted slightly (Fryk, 2007). Brynjolfsson (1993) argued that the impact of information technology is much more intricate than Solow seems to suggest. Brynjolfsson noted that the lack of visible impact of information technology could be explained by failure of current measurement methods used, as these do not always capture the actual impacts although they often exist. The explanation provided by Brynjolfsson is especially true when we consider law enforcement. Impacts of information technology in law enforcement are hard to define due to the complex nature of the inputs and outputs (Lind & Lipsky, 1971). An increase in information technology does not simply result in an increase of ‘law enforcement’, rather information technology and law enforcement is the “interaction between technology and people, with people having to make sense of it; and in this sense making process, they have to develop particular assumptions, expectations and knowledge of the technology, which then serve to shape subsequent impacts because of it.” (Orlikowski & Gash, 1994) Orlikowski (1994) therefore argues that the impact of technological change cannot be understood in a deterministic or rationalist way. Instead, the impacts of information technology should be measured through an interpretive or emergent (systems thinking) approach — that is, the impact of 14 Measuring the impact of information technology systems on police practices technological change cannot be simplified towards single parts but should rather be viewed as a whole that is subject to different contexts. Yet, a majority of over 80% of articles in leading IS journals still use the deterministic approach in conducting research (Burton-­‐Jones, Mclean, & Monod, 2011). Little utilization is found of the ‘Systems Thinking approach’ in IS literature. The intention of this research is to explore the complexity of technological impact on law enforcement, by coming up with a complete tool to measure this impact from a systems thinking perspective. This research will therefore have significant scientific value, since a systems thinking approach is not commonly used in IS literature. Using this approach might improve understanding of technological change and other technological phenomena such as the productivity paradox in a law enforcement context. Of course, this research also has practical value for law enforcement organizations themselves as it enables such organization to perform complete measurement of the police practices that are impacted by information technology. 1.3 RESEARCH OBJECTIVE As discussed in the introduction the impact of investments, implementation or even frequent use of information technology is not an obvious one. Therefore, it is important to map this impact of information technology. The development of an impact measurement method of information technology on law enforcement tasks, more specifically police practicing tasks, will be leading in this research. Furthermore, due to the complexity of information technology and law enforcement tasks, this research will adopt a systems thinking approach. How the systems thinking approach is applied to reach the research objective is discussed in the research design in 1.5. The following research objective can be derived: Develop a method to measure the impact of information technology on police practices Measuring the impact of information technology systems on police practices 15 1.4 RESEARCH QUESTIONS To accomplish the above stated research objective, the following main research question is formulated: How can the impact of information technology on police practices be measured? The answers of the following sub questions should lead to the answer of the main research question. 1) What is the domain of the research objective and the corresponding problems within this domain? 2) What is systems thinking and how can it be applied on the development of a measurement tool? 3) What are concepts that are relevant within the domain of this research? 4) How can these concepts be measured? The first question is related to finding definitions and identifying problems related to measurement of technological impact in a policing context. The problematic notion identified in the first question triggered the use of a system thinking approach. Therefore the second question will explain what systems thinking is and how it will be applied in this research. The third question focuses on identifying concepts relevant for measurement, while the fourth question focuses on concepts that will be included and operationalized in the measurement tool in relation to the belevingsmonitor. 1.5 RESEARCH DESIGN In this paragraph, short descriptions of the research perspective, object, outcome and framework are given. The subchapter’s 1.5.1 through 1.5.3 will discuss all these elements individually, where the research design as a whole should become clear. It should be noted that this research differs from ‘normal’ deterministic or reductionistic approaches and needs further explanations. This research uses system thinking methodologies such as ‘mental maps’, which will be explained in further detail in chapter two. For now, a brief description will be provided for terms used in the research framework illustrated below (figure 1.5-­‐1). A measurement tool always consists of a ‘what’ a ‘how’ to measure (Caballero, Verbo, Calero, & Piattini, 2009). The ‘what’s’ to measure, being the object of this study will be identified through the 1
mapping of a ‘mental model’ . The mental model will be mapped based on information derived from literature and qualitative empirical data. From the mental model combined with additional quantitative empirical data, the most relevant concepts eligible as measurement indicators will be identified. For practical purposes, this research will discuss the practical implications of the results of this research. In order to discuss these practical implications, this research will reflect upon the belevingsmonitor, the current measurement method in the police force, and propose several scenarios for adjustment of the belevingsmonitor. These proposed scenarios range from slightly modified versions of the belevingsmonitor to a complete revamp of the belevingsmonitor. 1
A mental model is a model of selected concepts and the relationships between them based on someone’s thought process (Forrester, 1973). It is a model of how someone thinks the world works, not how the world actually works. 16 Measuring the impact of information technology systems on police practices The needed information concerning mapping of the mental model is triangulated through literature and empirical data derived from the Dutch police force. The triangulation from both theoretical and empirical perspectives should avoid any biased information as much as possible. This approach was executed during a 7-­‐months internship at ICT organization of the Dutch police force under supervision and guidance of Drs. Ing. Ruud Hoefnagel, an internal researcher at the Dutch police force. The research design is illustrated in the operational research framework figured below. Perspective
Literature
Definition of IT
impact on police
practices
Qualitative
emprical data
Quantitative
empirical data
Belevings
monitor
Mental model
of IT impact on
police
practices
Relevant
indicators
Measurement
tool scenarios
How?
What?
Outcome
Object
Figure 1.5-­‐1: Operational research framework 1.5.1 RESEARCH OBJECT The objective of this research is to develop a tool to measure impact of information technology on law enforcement police practices. As illustrated in the research framework, this objective is achieved by mapping a mental model of the impact of Information technology on police practices. This mental model functions as the object of this research, which will assist in and lead to the identification of relevant indicators for measurement. This mental model is mapped based on analysis of literature and insights from qualitative empirical data derived from a case study. The relevant indicators identified from this mental model will result in the outcome of this research. 1.5.2 RESEARCH PERSPECTIVE There is sufficient literature available on the topic of IT impact. However, much of the approaches in these earlier studies are often deterministic and reductionistic in nature (Burton-­‐Jones et al., 2011). Thereby, these earlier studies consider one-­‐sided recommendations or very few variables in their studies. In contrast, this research considers a holistic systems thinking approach by employing the use of a mental model. The concepts and relationships used for the mapping of this mental model originate from multiple perspectives. The multiple perspectives that are considered in the mental model range from different sources of literature to the sampling of different stakeholders in the collection of empirical data (interviews, detailed in chapter three) 1.5.3 RESEARCH OUTCOME The main outcome of this research is recommendation of several measurement scenarios in perspective to the current design of the belevingsmonitor. 1.6 THESIS OUTLINE Measuring the impact of information technology systems on police practices 17 This report contains eight chapters divided over four major parts. The first part consists of this chapter (Ch. 1) and is written to provide clarifications of the research approach and the motivation to conduct this research. It provides background information, the research objective, research questions, and the research design. This first part functions as the introduction and overview of this research thesis. The second part (Ch. 2) of this research describes the relevant definitions, explanations and methodologies that will be used in this research. Chapter two focuses on the subject of this research, thus providing terms and associated problems when considering measurement of technological impact on police practices. The chapter follows up with an explanation of the systems thinking approach. Methodologies relevant for the systems thinking approach will be explained and motivation will be given as of why systems thinking was considered appropriate. The third part (Ch. 3 -­‐ Ch. 6) describes the system design and the corresponding results. Chapter three contains specific descriptions of exact processes that were used to design a mental map, identify indicators. It also includes, the data gathering methods used to design the measurement tool. In Chapter four presents the results of identified concepts and the constructed mental map. Chapter five presents the validation results. Chapter six presents the recommendations to the belevingsmonitor based on the mental map and identified indicators identified in this research. The final part is the conclusion (Ch. 7). This part is expected to generate an overview of the findings in this project and how these contribute to the already known. Also, in chapter seven the conclusions, limitations, reflections and suggestions for further research are given. 18 Measuring the impact of information technology systems on police practices 2. DOMAIN In this chapter a short overview is given of the domain of the research topic. The topic of this research is focused on the domain of information technology and police practices. Therefore, the topics related to information technology and police practices will be explored, discussed and whenever possible, defined. After providing definitions for the domain, this chapter continues with a description of the systems thinking perspective applied in this research. Starting from the origins of the systems thinking approach, the differences with other approaches it follows with a motivation and a generic description of how the system thinking approach is applied. 2.1 INFORMATION TECHNOLOGY Literature and practice use various terms such as information technology, information communication technology and information systems without explicitly defining the similarities and differences between them. Therefore, before going further with this research the definitions, relationships and differences between these terms will be given. An indication of how these terms will be used throughout this thesis will also be given. According to Pelgrum & Law (2003), the explicit distinction between information technology (IT) and information communication technology (ICT) originates from developments in technology throughout history. When microcomputers became available for the consumer market, the term ‘computer’ was introduced. “But nearing the end of the 1980s, the term ‘computers’ was replaced by ‘IT’ (information technology), signifying a shift of focus from computing technology to the capacity to store and retrieve information. This was followed by the introduction of the term ‘ICT’ (information and communication technologies) around 1992, when e-­‐mail started to become available to the general public”. When considering Pelgrum & Law’s distinction between IT and ICT, we can derive that the term IT encompasses computing technology focused on the processing of information. In contrast, ICT is a more encompassing term that reflects today’s use of computing technology, which extends from the information processing aspect of computing and to the communication aspect of computing technology. In strict definitions, ICT and IT are two different concepts. This distinction between ICT and IT would be relevant in the 1980’s and 1990’s, where technologies that enable information process computing or telecommunication could still be separate technologies. More recently however, the technologies that enabled information processing and telecommunication have converged with the emergence of the internet, network computing and IP based multimedia. Thus making the classical distinction between telecommunications and computing no longer useful (Messerschmitt, 1996). While strictly two different concepts, in this thesis we consider IT to be equal to the concept of ICT since we cannot make a strict exclusion of the communication part when implementing IT. What remains is the definition of the concept of an information system (IS) and how IS relates to the concept of IT. The relationship between these two concepts depends on the perspective that is chosen. (Fryk, 2007) sees IT as the broader concept to IS. He illustrates this by the following example, when Measuring the impact of information technology systems on police practices 19 considering an organization’s whole range of IT infrastructure, an information system such as an ERP (Enterprise Resource Planning) would only utilize a part of the whole organization’s available IT, therefore the ERP IS is a small part of the broader concept of a company’s IT. On the other hand, Alter (Alter, 1999) defines an IS as a holistic system that is bounded by all things necessary to process information. Therefore, an IS can be seen as the broader concept to IT. The tools used in an information system to process information can typically include information technology, but can also include other tools such as pen and paper. IT (ICT)
IT ∩ IS
IS
Term Information technology (IT) Definition The computing technology that is mostly used in the current information systems. Information system (IS) Information processing environments that mostly use IT, but can also include pen and paper. Figure 2.1-­‐1 Venn diagram of IT definitions Following from the perspectives given by both authors, the relationship between IT and IS can be viewed as illustrated in the Venn diagram below. When we refer to IT, we generally refer to IT that is 2
used in information systems (IT∩IS) . When we refer to IS, we can refer to systems that use IT (IT∩IS) but can also use other tools that do not fit the description of IT (right part white circle). We included descriptions of the terms in a table. In this research, we will use the terms IT and IS interchangeably because of the slight overlap in both concepts. We will use IT as a term when we want to emphasize on the technological aspects in an IS (IT∩IS in figure 2.1-­‐1); in other sentences we will refer to IS as a term when we want to emphasize the use of technology in the processing of information, without explicitly excluding non-­‐technological elements of an IS (IS in figure 2.1-­‐1). The last distinction we make is between the product that information technology or an information system produces, which is information. This information can either be in the form of digitized information flows but also physical information flows Carlsson (2004). Information becomes knowledge when information is interpreted and understood by humans Fryk (2007). We will employ this distinction between information and knowledge. 2.2 TECHNOLOGICAL IMPACT Impact is an ambiguous term(DeLone & McLean, 1992). Information technology can impact many things and due to this ambiguity, authors often use the term impact in their research to denote very different levels of analysis when conducting research regarding the topic of technological impact (Surry, 2008). 2
We assume that IT that is not used in IS does not exist, therefore we indicated this part grey 20 Measuring the impact of information technology systems on police practices Because of the ambiguity on the word impact and various levels chosen by authors, it is important to explicitly define what level of impact analysis is considered in this research. In this research we have analyzed the various levels of impact analysis defined by different authors. Altough many authors (Alter, 1999; Cyrus, 1991; Delone & McLean, 2003; Gable, Sedera, & Chan, 2008; Heejin Lee & Whitley, 2002; Mason, 1978; Torkzadeh & Doll, 1999) provided various ways of categorizing impact analysis levels, we found that most categorizations of impact analysis levels generally fall in the three levels of analysis defined by Delone & Mclean (2003) (more information on these categorisations can be found in appendix G). According to Delone & Mclean (2003), impact analysis can generally be conducted on three levels: analysis of IT impact on a systematic level, analysis of IT impact on the level of understanding use and analysis of IT impact on the consequences of use. system creation
system use
consequences of use
Individual
Organisational
Society
Figure 2.2-­‐1 Levels of impact measurement The subject of this research is the impact of information technology on police practices. Police practices in this research are considered to be all the activity conducted by individuals that make up police practices. Therefore, this research focuses information technology impact analysis on the level of consequences of use, more specifically consequences of use on individuals. Following Delone & Mcleans categorization, the level of impact analysis for this research is marked by a white box in figure 2.2-­‐1. 2.3 COMPLEXITY OF IMPACT ON POLICE PRACTICES As discussed briefly in the introduction, this research follows a systems thinking approach due to the complexity of the interaction between technology and police practices are. This section provides brief explanations as of the cause of this complexity, from the point of view of police practices and technological change. 2.3.1 COMPLEXITY OF POLICE PRACTICES BEING IMPACTED In order to be able to measure impact of IT on police practices, a measurable definition of the concept of police practices is necessary. What are police practices? . According to Lind & Lipsky (1971) finding such a definition for police practices is not easy. Three main hypotheses can be given to explain the difficulty of defining the term police practice. These are given below. Police practices are difficult to define as a singular concept Lind & Lipsky (1971) argued that an attempt of defining the concept of police practices is a question of infinite difficulty. Ideally, the concept of police practices would consist of a singular theoretical measure that is capable of measuring the output of police practices on society as a whole. Such a singular theoretical measure that would represent police practices does not exist. Even if such a theoretical measure would exist the problem remains on how to define such a theoretical concept. Police practices cannot be just substituted by a simpler concept Measuring the impact of information technology systems on police practices 21 The problem with defining the concept of police practices, as a whole is that it cannot be captured by a singular concept (Lind & Lipsky, 1971); it is the accumulation of many practices that interrelate with each other. The very nature of police practices is extremely complex in today’s society. Police officers play an important role comprising of many different tasks concerning; actually enforcing the criminal law, performing order maintenance and other miscellaneous services. It is through these duties and services that police are constantly intersecting and interacting with the community on a daily basis. CRIME RATES Simplifying the concept of police practices into simpler substitute concepts would also not do the concept of police practices justice. Examples of such substitute measures are crime rates (Lind & Lipsky, 1971). Crime rates are easier to understand and to define as the number of criminal offenses. PRODUCTIVITY Another traditional substitute concept is to consider the concept of productivity, but having this view can cause problems. A focus on productivity output can often leave out benefits such as quality of the product (e.g. Brynjolfsson & Hitt, 1996a, 1996b, 2000). Measuring productivity output often misses the fact that many returns materialize over time (e.g. Devaraj & Kohli, 2003a, 2003b), get redistributed over other outputs or are dissipated by other influential factors. Different stakeholders perceive the concept of police practices differently “Individuals with different backgrounds and philosophical positions will subscribe to different objectives and meanings of a concept, and no one definition will command universal acceptance”(Lind & Lipsky, 1971). Different types of individuals will subscribe different meaning to the concept of police practices. A operative police officer will define police practices of certain concepts that would be totally different than how a detective will define police practices. 2.3.2 COMPLEXITY OF TECHNOLOGICAL CHANGE In order to explain the complexity of technological change, an explanation is needed to describe how emergence differs from a linear determinism. Therefore, an explanation of a linear deterministic model is given first. One of the first (theoretical) frameworks developed for historically understanding technology and its impact has been the “linear model of innovation”. The Linear Model of Innovation is an early model of innovation that suggests technical change happens in a linear fashion from Invention to Innovation to diffusion. Development à production and diffusion à impact While the definition of impact suggests this linearity of technological impact, in reality technological impact is much more complex. The linear model of innovation has therefore been criticized for over-­‐
narrowly equating research and development with technological change (Shapira & Rosenfeld, 1996). As claimed by N. Rosenberg and others (1994) “everybody knows that the linear model of innovation is dead”. In the context of understanding the impact of information technology on policing, cannot isolate the impact of information technology to the physical, material dimension of it (Chan et al., 2001) because Technology operates in a social context (Manning, 1992). To interact with technology, people have to make sense of it; and in this sensemaking process, they develop particular assumptions, expectations and knowledge of the technology, which then serve to shape subsequent actions toward it. (Orlikowski & Gash, 1994) 22 Measuring the impact of information technology systems on police practices Technological change and thus its impact does not emerge from the unfolding of a predetermined logic or a single determinant, different routes are available, potentially leading to different technological outcomes (Williams & Edge, 1996). A more modern view on innovation that goes beyond the linear model and stresses the complex character of technological change is therefore necessary. Orlikowski (1996) has therefore argued that the impact of “technological change on work practices, communication channels and performance cannot be understood in a deterministic or rationalist way” (Orlikowski, 1996). Instead, the consequences of information technology should be interpreted via an interpretive or emergent model — that is, that they result from the ‘interplay among computing infrastructures, conflicting objectives and preferences of different social groups and the operation of chance’ and that information technology is open to interpretation during implementation and use (Robey & Sahay, 1996). Based on these finding, this research adopts the following notion of complexity: Due to the emergent property of technological change, it is impossible to isolate concepts of police practices impacted by technology alone. 2.4 SYSTEMS THINKING Before motivating and describing the system thinking approach used in this research, this section will start off firstly by discussing the origins of system approach and definition of concepts such as systems and mental maps. 2.4.1 ORIGINS OF SYSTEMS THINKING During the early 1950s, science was becoming increasingly more fragmented and diverse fields of research were showing duplications of scientific and technological research. As a response to this parallelization of science, the allgemeine Systemlehre (general system theory) was introduced by Bertalanffy (1950). Although this idea was already presented in 1937 in a philosophic seminar at the University of Chicago, it was first published after World War II. Accordingly, the world of the 19th and the first half of the 20th century was conceived as a world of chaos. The theory tried to structure this chaotic earth and looked for another basic outlook on the world, the world as an organization (Laszlo & Krippner, 1998). It is a logico-­‐mathematical discipline and applicable to all sciences concerned with systems. In order to do so, different scientific and technological disciplines were integrated and mend to function as the generalized ears. No longer should economists solely talk to economists, physicists talk to physicists, etcetera. The more science breaks into sub-­‐groups and the less communication is possible among the disciplines, however, the greater the chance there is that the total growth of knowledge is being slowed down by the loss of relevant communications. Therefore, according to Boulding (1956) the general systems theory is the skeleton of science. This framework or structure of systems should be used for the hanging of the flesh and blood of other disciplines to form a corpus of knowledge. Furthermore, the use of the General Systems Theory is meant to shift ‘between the platonic world of theory and the fuzzy world of practice’. Any user of the theory should balance between being too specific that has no meaning for any general discovery and being too general that has no content for any application. For each purpose and level of abstraction, the goal is to reach an optimum degree of generality (Boulding, 1956). Measuring the impact of information technology systems on police practices 23 This development of the General Systems Theory leaded to a new approach in science. The world, and all it contains, is a collection of distinct but small parts which are largely fit for study and analysis and isolation. Therefore, instead of focusing on each individual part and its integration, the system is investigated in a way in which the position of the parts becomes irrelevant. It focuses on the whole as well as on the interrelated parts of the system. This approach is not an alternative to the former reductionistic approach but a complement (Laszlo, et al., 1998). The exact origin of this type of thinking is difficult to appoint. However, an important event in the history of Systems Thinking is the foundation of the Systems Dynamic Group at MIT’s Sloan School of Management by professor Jay Forrester in 1956 and the early disclosure of System thinking ideas of von Bertalanffy, Boulding, Whitehead and Weiss. Later on, among others, scientists from MIT further developed this theory. 2.4.2 SYSTEMS A system, or “a collection of interacting components organized to accomplish a certain function or set of functions within a certain environment” (IEEE, 1998),requires a different approach than Descartes’ traditional reductionistic approach. Treating key events, issues or symptoms by a one-­‐dimensional mindset of isolation and analysis is often not sufficient to handle the complexity of most problems. Reductionism can be victorious when dealing with technical, straightforward problems but will, most likely, not be very successful for solving complex problems. To understand and manage complex problems, the Systems Thinking approach should be applied for understanding fundamental causes and creating a sustainable solution. This scientific methodology focuses on the whole system as well as the constituent components and their interactions (Burton-­‐Jones et al., 2011) This holistic approach focuses on the tree and the forest instead of only one tree individually. To handle such complexity, the human brain is often too cognitive limited. Therefore, in the field of Systems Thinking, a variety of diagrams, graphs and computer simulation models are used to assist in illustrating, understanding the system and predicting the system behavior. Unfortunately, in the field of IS literature however, this ‘new way of thinking’ to understand and manage complex topics on information technology (Burton-­‐Jones et al., 2011) is still relatively new. This study is one of the first to address issues and challenges related to information technology impacts and policing using a Systems Thinking approach. For that reason and because of the size of the global field of IS literature, it would have an high potential to further research this new field of science. 2.4.3 MENTAL MODELS System thinking is trying to see the forest and the trees at the same time. In order to see these, we need to look at the forest from a satellite. This means that systems thinking starts with drawing a picture of the system as big as reasonably possible. Mental models are one of the instruments that assists in gaining this satellite perspective. To understand the concept of the mental models, the ‘Four levels of thinking’-­‐model (Maani & Cavana, 2007) should be explained. These four closely related but distinct levels are (1) key events, symptoms or ‘the tip of the ice-­‐berg’; this level represents the daily activities which is the most visible and shallow level of reality, (2) patterns of behavior; the recognition of repetitions of a larger set of events creating a history, (3) systemic structures; the disclosure of how the patterns relate among and with each other and how components interact, and lastly, (4) mental models; in this deepest level of thinking, the personal believes, assumptions and underlying values for any reasoning or motivation are stored (Maani & Cavana, 2007) 24 Measuring the impact of information technology systems on police practices Unfortunately, in reality, most research on technological change is focused on tips of the icebergs instead of the whole iceberg. In order to achieve sustainable understanding of the structure of the system, a collective mental model of the current police practices system impacted by information technology should be mapped. A mental model is thus a mental representation of how individuals perceive reality to be. The more people involved in this process, the reliable the mental representation of reality will be. The process of mapping such a mental model assists us in visualizing and understanding the structure of the forest and trees. This means that the construction of a mental model is never meant for predicting and actually representing the factual real world. In this research, literature and interviews are used for the construction of a mental map. As the construction of the map continues and more interviews are conducted, the more the map should represent the mental models of the police practitioners. 2.5 APPLICATION OF SYSTEMS THINKING APPROACH Before describing the systems thinking framework describing the approach used in this research, first a motivation for the system thinking approach is given. 2.5.1 MOTIVATION FOR SYSTEMS THINKING As described in the introduction, this research adopts a systems thinking approach in developing a measurement tool for the impact of information technology on police practices. This section provides an explanation as of why this approach is adopted in this research and its suitability compared to other approaches. In the field of IS literature there is prevalent use of theoretical models, either in the form of dedicated IS models or in forms adapted, simplified or extended from other research disciplines. According to (Burton-­‐Jones et al., 2011) a majority of these models used in IS literature roughly adopt two research approaches, which are variance approach and the other being a process approach. The variance approach is one of the most popular theoretical approaches in IS literature, with over 80% of articles in leading IS journals to have used the variance approach (Pare et al. 2008). Popular examples of variance approaches are models such as TAM, UTAUT and SERVQUAL. The variance approach is based on a worldview aimed towards mapping the variance of independent factors that predict a dependent factor. These variance models follow basic causal, deterministic logics: the precursor or the cause is posited as a necessary and sufficient condition for the outcome (Markus & Robey, 1988). Markus & Robey illustrates this logic by the following example. If a variance model hypothesizes use of information technology as a cause of organizational centralization, centralization is expected to always occur whenever information technology is used. 20% of the remaining articles found in leading IS journals used a process approach (Paré, Bourdeau, Marsan, Nach, & Shuraida, 2008). In contrast to the variance approach, the process approach does not assume that a certain outcome will occur invariably if any necessary and sufficient conditions are present. Instead, the process approach assumes that outcomes occur based on necessary conditions in sequence along with chance and random events playing a role (Markus & Robey, 1988). One of the popular examples of process-­‐approached models is the IS-­‐success model by Delone & Mclean. In this example, system use is a necessary condition for individual impact to occur because without system use, there can be no consequences or benefits. System use however, does not necessarily predict individual impact (Delone & McLean, 2003). Measuring the impact of information technology systems on police practices 25 Although the systems thinking approach is uncommon in IS journals, this research believes that it is the most appropriate approach in developing a measurement tool. Below follows a motivation as of why the systems thinking approach is considered most appropriate. This is done by providing illustrations of the different approach perspectives first and providing argumentation afterwards Variance approach In a variance approach to information technology impact on police practice the visualized modeling perspective would look as shown in figure 2.5-­‐1. In a pure variance approach, the impact of information technology is viewed as influencing or even predicting change in several dependent factors, which in its place predict the occurrence of the factor ‘police practices’. Process approach A process approach requires a model’s concepts be events that follow a probabilistic sequence. From a process approach, the impact of information technology could be viewed as impacting a series of factors that altogether Information
technology
Police
practices
Figure 2.5-­‐1 A variance approach conceptualization Police practices
Information
technology
Figure 2.5-­‐2 A process approach conceptualization comprises the concept police practices. Systems thinking approach Finally a systems thinking approach is visualized in the figure 2.5-­‐3. A systems approach requires that a model’s concepts involve interacting parts and emergent properties. Figure 3.2-­‐3. shows how impact of information technology on police practices is perceived if it followed a systems approach. The figure shows that information technology and police Police practices system
Information
technology
practices are systems of which the Figure 2.5-­‐3 A systems thinking approach conceptualization information technology (system) influences the police practices system. Information technology thereby influences police practices as a whole where each part within the police practices system are all influenced. Moreover, interactions within between the parts in the police practices systems results in an emerging change of the police practices system as a whole. ARGUMENTATION Inspecting how technological impact on police practices is conceptualized following the three approaches does not necessarily entail that some approaches are wrong or right. What these conceptualization do show however, is that the system thinking approach seem to respect the 26 Measuring the impact of information technology systems on police practices characteristics of police practices and technological changes as described in chapter two. As the illustration shows, system thinking does not consider police practices as a singular concept or a concept that can be substituted by a multiple simpler concepts. Instead system thinking considers police practices to consist of a system consisting of a large amount of concepts interacting with each other. System thinking also respects the fact that technological change cannot be isolated to the few or parts of the system and rather considers information technology to impact the system as a whole. 2.5.2 SYSTEMS THINKING FRAMEWORK Identify concepts
Mental map
Identify most relevant indicators
Propose measurement scenarios
Figure 2.5-­‐3 Systems thinking approach framework For the development of a measurement tool, the system thinking tool approach will be applied as follow: 1.
2.
3.
4.
This research assumes that the whole system is impacted by information technology. Therefore, relevant concepts representing the police practices system will be identified. In order to improve understanding a mental map of all the identified concepts and its potential relationships are mapped. Based on the map and additional quantitative data, the most en least relevant indicators within the mental map will be identified. The identified indicators will be reflected to the belevingsmonitor and several measurement scenarios will be proposed. 2.6 CHAPTER CONCLUSION This chapter described the domain of this research and follows up with a history, motivation and generic description of the system thinking approach applied in this research/ Definitions were given to the terms information technology and an impact level of analysis is chosen, being the analysis of impact on the consequences of individual use. While information technology and impact levels are relatively simple concepts to understand on its own, the combination of technological impact and police practices is much more complex in its definition. As discussed in this chapter, many aspects contribute to the complexity of technological impact and police practices. The properties that attribute to this complexity is listed as following: •
Police practices are difficult to define as a singular concept •
Police practices cannot be just substituted by a simpler concept •
Different stakeholders perceive the concept of police practices differently •
Due to the emergent property of technological change, it is impossible to isolate concepts of police practices impacted by technology alone. Considering these properties, dividing the subject of police practices into individual sub-­‐parts would be a reductionistic approach, which probably won’t help understanding of technological impact and Measuring the impact of information technology systems on police practices 27 police practices as a whole. When dealing with such a complex subject, the subject should be seen as a system. Therefore, a holistic approach is needed which encounters many issues simultaneously. In order to explain what system thinking is, a brief historical description is given of how system thinking became to exist. Concepts relating to systems thinking were explained, such as the concept of a system itself but also the use of mental maps as a tool in a system thinking approach. The second section focused on motivating as of why system thinking is considered the most appropriate approach in developing a measurement tool. An illustration of three most common research approaches (variance, process and systems thinking) were given followed by an argumentation for the appropriateness for a system thinking approach. The third part concluded with a systems thinking framework that explains how the tools of mental mapping and identifying high relevancy indicators will contribute to the development of a measurement tool. In chapter three, this system-­‐thinking framework will be put into practice by designing a mental map of police practices. The results of this system design process are then presented in chapter four. 28 Measuring the impact of information technology systems on police practices 3. SYSTEM DESIGN This chapter describes how the system thinking approach is applied to come up with a measurement method for the impact of information technology on police practices. In the earlier chapters the methodology and the motivation for this approach is described. Chapter 4 focuses on the actual design of the ‘system’, which comprises of a mental map. Starting point of the system design is the construction of a literature-­‐based model. This model should evolve with input from the empirical analysis into the mental map that should eventually lead to the identification of most relevant concepts for measurement. The development model describes the eight iterative steps including the verification and the validation of the system. The outcome of this development model is a applicable measurement model to gain insight in the impact of information technology on police practices. The spiral model can be executed multiple times to strengthen the system. This chapter is written from the perspective of the researcher. 3.1 STARTING POINT IN LITERATURE In this research there are three reasons as why to use literature as a starting point in system design. The first reason is to develop a measurement method that is grounded in theoretical literature. The second reason is simply not ‘having to reinvent the wheel’. If concepts and models are provided by literature, these concepts and models should be adopted as much as possible in the system design process. Thirdly, concepts and models provided by literature can serve as an anchoring point for the construction of the mental model. The use of literature is done by means of a systematic review. For the systematic review on the topic of impact is done by searching Google, Google scholar, Scopus, Web of Science, IEEE and Jstor for keywords such as information system(s), IS, impact, acceptance, technology, success, police and law enforcement as additional keywords. The search will also extend to the use of keywords derived from internal references found in relevant articles. Articles will be considered relevant if the author of an article acknowledges the emergent property of information technology impact. As discussed in chapter 3, a great majority of articles published in IS-­‐journals follow a reductionistic approach on information technology impact. Therefore the expectation is that only a small amount of articles will be found useful as a starting point for the system design. 3.2 DESIGN PROCESS As discussed in the previous chapter, this research applies a Systems Thinking approach primarily by utilizing the systems perspective to develop a mental map. The methodology of the system design primarily followed a sequential waterfall design process, with the exception of the interviews, interview establishment and identification of concepts. During these three steps many back and forth iterations were conducted to adjust interview structure and respondents based on preliminary identified concepts. Nevertheless, this section discusses the design process in a linear way. The design process consisted of eight steps, which are interviews, establishment of interviews, identification of concepts Measuring the impact of information technology systems on police practices 29 3.2.1 STEP 1: INTERVIEWS The purpose of the interviews is to verify the concepts found in literature with the statements provided by a selection of stakeholders. This switch from theory to a practical setting is useful for the later mapping of the mental model. The statements resulting from these interviews will be used to contradict, confirm or add to the concepts from literature. We choose to implement the interviews in participatory, observational settings at an interviewee’s direct working place. The direct interaction between interviewee and interviewer in such settings provides room for direct adjustment, discussion, explanation and discovery of topics directly during data collection. This setting also allows them to demonstrate or provide direct observable examples to illustrate their statements. The setting also allows for spontaneous statements based on occurrences of events. INTERVIEW PROTOCOL Each observational interview follows a narrative approach structure (Bauer, 1996). This means that the course of the interview will be semi-­‐structured and facilitative to enable the informant to tell ‘his story’. This narrative approach fits well with the observational interview focus. Informants are thereby able to tell their stories by directly demonstrating examples on their computers or working space. The narrative approach is also chosen deliberately, since it is undesirable to steer the interviewee into a confirmation bias, leading to interviewees confirming a checklist of concepts and the danger of not identifying new concepts. The narrative approach proposed by Bauer (1996) contains the following phases: 1.
2.
3.
Initiation: Introduce intention of interview, which is gaining knowledge on police practices and how IT supports it. Main narration: Let interviewee tell his story without interruptions until the interviewee stops his story. Questioning phase: Ask questions related to the story to further promote interviewee to provide further details. The question could be based on further explanation of tasks and/or how IT is/could be used for the task. Possible questions: •
Introduce possible new features of the new IS that may trigger narration •
Introduce statements made by previous interviewees An example of how the questioning phase could be conducted: Example: Currently, you are using paper printouts of your current tasks. These paper printouts will become digital with the new IS and accessible on your mobile phone. On the road and during patrol duty, how will this new digital output of execution task influence your job? Possible answer 1: I think that I will become more productive; I will be able to perform more of certain tasks Possible answer 2: With the digital printout I think the reception information will be more accurate and precise. Sometimes, paper printouts lay in on the office for a whole week. During that week a lot can happen and I can imagine that digital printouts will contain more accurate and up to date information. 30 Measuring the impact of information technology systems on police practices Through an iterative process each subsequent interview will be slightly adjusted, dependent on the outcome of previous interviews. After each subsequent interview the proportion will shift from providing room for narration towards asking more specific questions. It is expected that in the later interviews more data from previous interviews will become available, enabling the use of this data in confronting interviewees in later interviews for discussion. Thus, in later interviews less time will be spent on explaining generic topics and more on asking specific questions verifying previously stated statements. The execution of interviews will be ended at the moment of theoretical saturation. The moment of theoretical saturation is reached when outcomes of interviews become consistent; no new statements can be identified or verified. After this the statements are documented following the claims, concerns, issues data management technique. INTERVIEW DOCUMENTATION The statements made in interviews need to be documented. Therefore the claims, concerns and 3
issues documentation method is used. The method is applied as follows: •
•
•
•
•
Identify claim, concern or issue (definition of claim, concerns, issue given in table xxx below) Write keywords related to this claim, concern or issue. Provide a label keyword for this claim, concern or issue. If possible, assign a label derived from concepts in literature. Confirm this label to the informant during interview. Write out each claim, concern or issue after interview. Claims, Concerns & Issues (Koch 1994) Claims A claim is any assertion that a stakeholder may make that is favorable and is a platform from where the team can take the next steps. For example it might be that the staff feel they already have good teamwork on the ward.. Concerns A concern is any assertion that is unfavourable. The purpose is to highlight potential barriers: personal, systematic or organizational, real or perceived. An example might be that a collaborative nursing model will require a change to the way the rosters are usually structured. Issues An issue is a reasonable question raised through better understanding of claims and concerns. Issues are drawn from claims and concerns using What and How questions and issues will need to be addressed before progressing with the project. Table 3.2-­‐1 Definitions of claims, concerns and issues Writing down quick keywords enables documentation of the interviews without the necessity to record the interviews. Alleviating the necessity to record interviews enables to have longer, more flexible and informal interviews with the occasional interruption and spontaneous reactions. To illustrate, we conducted an observational interview during an on location traffic control for five hours. We conducted an interview directly behind an officers working desk, with his colleague giving the occasional comment in between. During one interview, a civilian showed up to complete a payment, 3
From initial interviews we observed a pattern in how informants formulated their narratives. These informants often spoke from the perspective of concerns or issues that they have related to IT or IS and their work. Based on this observation, we chose to document the interviews following the claims, concerns and issues technique. Measuring the impact of information technology systems on police practices 31 of which we interrupted the interview for. When the interview was resumed, we picked up on the issue of payments. 3.2.2 STEP 2: ESTABLISHING INTERVIEWEES As in every field research, more empirical data, or in this case a high number of interviewees, should give more representative and reliable end results. Yet, due to the limitations of time and resources, this research aims to have at least 10 different stakeholders interviewed, varying from operative police officers, coordinating police officers and administrative workers. The interviewees that qualify for data collection should be end-­‐users of IS, meaning that they should at least have hands-­‐on experience with the current IS. Experts and developers therefore do not qualify, since experts and developers are not directly involved with either police practices or use of these IS. It is also important to gather stakeholders from different geographic areas, since each department has substantially different working methods and processes (Wessels & Timmermans, 2013). The selection of the first interview will be sampled based on convenience. Most of the initial contact will be made by phone, e-­‐mail and visits. Thereafter, snowballing will be applied. This means that each subsequent interview will be selected based on recommendations made by previous interviewees. Sampling of interviews will be stopped when theoretical saturation is reached, meaning that interview sampling stops when interviewees provide no new statements. 3.2.3 STEP 3-­‐4: IDENTIFYING CONCEPTS AND MENTAL MAP DESIGN The construction of a mental map assists in illustrating and understanding of how the police practices system impacted by technology looks like. For the construction of the mental map, the concepts to be mapped need to be identified first before mapping the actual mental map. The following procedure was followed for determining the mental map that comprises the police practices system. Two tasks are used in identifying quality attributes of a product: (1) identifying concepts that comprise the police practices system, (2) identifying the relationships between those concepts. The identification of concepts was preceded by a literature review as a starting point (4.1) and the collection of statements (step 1-­‐2). (1) IDENTIFYING CONCEPTS Combining insights from literature and the provided statements from interviews should result in a list of concepts that altogether comprise the police practices system (being impacted by information technology). The inclusion of relevant concepts within this list is based on the following conditions: •
•
Concept matches the definition of individual impact in chapter two and Concept or term was mentioned at least once by authors from literature or at least once in the interviews Note that the focus of the concepts identification is oriented on completeness and richness. This means that the list should include as many concepts from literature and practice. This means that the list will be non discriminatory when considering the amount of references in scientific articles and/or stated in interviews. It is strived to comprise a list with concepts containing terms that were derived from literature and not invented by the author of this research. However, if such descriptive terms for certain concepts cannot be found in literature, a keyword from the statements will be used to describe the concept instead. 32 Measuring the impact of information technology systems on police practices (2) M APPING R ELATIONSHIPS By mapping relationships between the different concepts identified, the structure of the mental map will become clear. The mental map is constructed from the various insights derived from literature, but mostly the interviews conducted to identify concepts. Since interviewees will provide narratives and examples on how certain concepts relate to each other, the mental maps will be constructed based on these narrated relationships. The result of this process is a mental map of the police practices system that is impacted by information technology. As the word mental suggest, a mental map is a projection of all information that have been captured in a researchers mind (illustrated in figure 4.2-­‐2). Therefore, note that the documentation of how the mental map was constructed with be conducted on a general level. For this research, these generic descriptions should be sufficient considering the goal of the mental map. The mental map is to provide assistance and illustration of how the police practices system would look like and not to make an exact real world representation of the real world. Police practices system
Literature
Qualitative
emprical data
Figure 3.2-­‐1: Mental mapping as cognitive process 3.2.4 STEP 5-­‐6: VERIFICATION AND VALIDATION In this research a distinction is made between verification and validity. Verification efforts are focused on checking the internal consistency, transferability and credibility between the interviews and literature research that were conducted to construct the mental map. Validation efforts on the other hand, are more focused on the checking the broader external applicability and generalizability of the mental map and the insights derived from. VERIFICATION According to Creswell (2012), in qualitative research, the goal is to achieve, as good as possible, understanding and deep knowledge of some social setting or phenomenon. Striving for understanding requires spending extensive time in the field. He believes that verification is critical to evaluating the quality of qualitative research. He identifies eight procedures for verifying qualitative research findings and recommends that any research study employ at least two of these procedures: This research employs the following four out of the eight procedures proposed by Creswell to provide verification of this research: •
Prolonged engagement Spending sufficient time in the field to learn or understand the culture, social setting, or phenomenon of interest. This involves spending adequate time observing various aspects of a setting, speaking with a range of people, and developing relationships members of the Measuring the impact of information technology systems on police practices 33 •
•
•
culture. Development of trust facilitates understanding and co-­‐construction of meaning between researcher and members of a setting. Ensuring prolonged engagement is achieved through engagement in observational interviews as described in the step 1. Triangulation Triangulation involves using multiple data sources in an investigation to produce understanding. This research performs literature research, qualitative empirical data and quantitative empirical data in order to achieve triangulation. Member checking Testing interpretations and conclusions with members of those groups from whom the data were originally obtained. This can be done both formally and informally as opportunities for member checks may arise during the normal course of observation and conversation. In this research member checking was conducted with respondents at both the interviews and sorting sessions. Rich, thick description Describing a phenomenon in sufficient detail one can begin to evaluate the extent to which the conclusions drawn are transferable to other times, settings, situations, and people. This research achieves this by the extensive documentation as discussed in step 1. The documentation of these rich descriptions are presented in identified concepts described in section 4.1 and also in appendix A. VALIDATION Validity, in qualitative research, refers to whether the findings of a study are true and generalizable. Validation is the process of determining the degree to which a model is an accurate representation of the real world from the perspective of the intended uses of the model. Using data from the sorting sessions the validity will be assessed of the following: •
•
•
Validity of identified concepts Validity of mental map Validity of theoretical assumptions of system thinking perspective o Validity of dual perspective o Validity of assuming complexity and emergent nature of policing and technological change Why perform relative ranking instead of absolute ranking? As explained system thinking approach, the assumption of this research is that we cannot isolate concepts that are solely impacted are not impacted by information technology alone. Rather information technology affects the whole ‘system’, in this case the whole police practicing system. Therefore absolute relevancy of a concept does not really exist. Every concept within the police practices system is influenced some way or another, but it is more interesting to find out which concepts are influenced the most and which concepts are influenced the least. 3.2.5 STEP 7: IDENTIFICATION OF MOST RELEVANT INDICATORS The identification of most relevant indicators enables the development of a sparse measurement model. By focusing measurement on the most relevant indicators instead measurement of all relevant concepts, resources necessary to use the measurement model will be taxed to a lesser extent (shorter survey durations or less observations). 34 Measuring the impact of information technology systems on police practices For the identification of most and least relevant indicators, we collected data by letting end-­‐users sort the various measurable concepts identified in step 3 based on their personal interpretation of relevancy. In order to reduce the cognitive load of this sorting process, instead of asking end-­‐users to compile a top 44 from 44 statements, we applied a 3x3 sorting methodology. Users were asked to sort statement cards in bins of three being (high relevancy, medium relevancy and low relevancy). After sorting all cards within the three bins, users were asked to sort once again in three bins. This eventually results in datasets with one to nine ratings for each measurable concept. An important criteria for the identification method, is that it should force users to make explicit decisions as to which concepts should be considered most and least relevant. We want to avoid the risk of users rating all measurable the same. Thus making the data useless for the identification of most relevant concepts for measurement. The advantages of this sorting method compared to a regular 9-­‐scale survey are the following: •
•
A 9-­‐scale survey forces users to choose a reference point right from the first concept to be rated. The 3x3 sorting method instead assists users in gaining an overview of all the available cards by sorting in stacks of three first. After the first sorting phase, users can still drag and drop concepts from one stack to another. This increases the ability for users to conduct relative comparison. Because the stacks of three provides visual encouragement for each stack to be filled with cards, users are expected to make more distinctive decisions in placing cards between the 3x3 boxes (forced distribution). This implicitly encourages users to give more distinctive ratings compared to a 9 scale survey. 9 scale survey rather encourages central tendency in choosing a scale. Do not however, that there are no limits to the amount of cards that can be placed in a stack. We even explicitly mentioned respondents not to make even stacks and just to drag cards according to their own desire. By combining insights from the mental map and from the ranking method as described above, the indicators proposed in measurement scenarios will be determined qualitatively. 3.2.6 STEP 8: PROPOSING FUTURE MEASUREMENT SCENARIOS The insights derived from the mental map and the above identified most important indicators will be used to make recommendations towards future measurement of technological impact on police practices. Since the police forces already utilizes a measurement tool called the belevingsmonitor, this research will propose several scenarios with each scenario containing gradual adjustments propositions for the belevingsmonitor. 3.3 CHAPTER CONCLUSION In this chapter, the design and the propositions towards a measurement tool following a systems thinking approach is described. The proposition towards a measurement tool evolves from identifying concepts relevant for measurement from literature and interviews and a mental map to a list of most and least relevant indicators. Eventually, this research will recommend several scenarios for the design of a measurement tool in relation to the current measurement tool, the belevingsmonitor. In chapter four, the designed system will be discussed, which is the design of the mental map. Chapter five will provide validation results and lastly, chapter six will present the recommendations and proposed scenarios for measurement. Measuring the impact of information technology systems on police practices 35 4. MENTAL MAP OF THE POLICE PRACTICES SYSTEM This chapter describes the results of the identification process and presents them in a mental map. The results presented in this chapter follow parts of the process described in chapter four. First, an introduction is given on how concepts were identified. Then the identified concepts from the interviews will be given. In order to visualize how all these identified concepts relate to each other in the police practices system, a mental map is presented in the final section. Although the mental map itself is not required for the development of a measurement tool it does provide visual guidance in choosing indicators for inclusion in the development tool. 4.1 IDENTIFICATION OF CONCEPTS FROM EMPIRICAL DATA The presentation of identified concepts will be in two parts. Section 5.2.1 presents the identified concepts in the form of a narrative, but will start with some background information first regarding the context of the interviews. Section 5.2.2 will present the identified concepts once again, but in the form of a table overview. 4.1.1 FINDINGS FROM INTERVIEWS FROM INTERVIEWS TO IDENTIFICATION OF CONCEPTS As described in chapter 4, this research conducted interviews in an observation setting where respondents could provide narratives based on demonstrations and hands on examples. Instead of documenting whole interviewing sessions, this research only focused on documenting claims, concerns and issues (Koch, 1994) expressed by respondents. During the interviews a label and few keywords were written down for each claim, concern or issue expressed by the respondent. At the very end of the interview, we went through the notes that were made and asked for confirmation whether the respondent agreed with the labels that were given and whether certain claim, concern or issue were missing. After the interviews, the notes were written out in detail. This written out claims, claim or concern are included in appendix I. After writing out these claims, claim or concern we tried to match these claims, claim or concern to labels we derived from literature. Mainly three scientific articles and two government policing reports were used to derive labels from. This matching process was an iterative process, instead of having a checklist of labels from literature; gradually more labels were found in the five sources to match the claims, concerns and issues expressed in the interviews. The results of this matching process have been documented in appendix A and B. Appendix A contains a detailed documentation of each claim, concern and issue sorted based on labels. Appendix B contains a checklist documenting from which sources certain labels originate. 36 Measuring the impact of information technology systems on police practices Claims, Concerns & Issues (Koch 1994) Claims A claim is any assertion that a stakeholder may make that is favorable and is a platform from where the team can take the next steps. For example it might be that the staff feel they already have good teamwork on the ward.. Concerns A concern is any assertion that is unfavourable. The purpose is to highlight potential barriers: personal, systematic or organizational, real or perceived. An example might be that a collaborative nursing model will require a change to the way the rosters are usually structured. Issues An issue is a reasonable question raised through better understanding of claims and concerns. Issues are drawn from claims and concerns using What and How questions and issues will need to be addressed before progressing with the project. Table 4.1-­‐1 Definitions of claims, concerns and issues LITERATURE USED FOR CONCEPT LABELS The five articles used to derive labels from are the result of a literature review of more than 51 sources. These articles are (Chan et al., 2001; Delone & McLean, 2003; Gable et al., 2008; Groff & Mcewen, 2008; Torkzadeh & Doll, 1999). There are three reasons why only five sources of literature were usable to derive concept labels from. The first reason is because these five articles acknowledged the richness of what IT impact is and why it cannot be a represented single concept. Therefore, these five articles provided a broad range of labels relating to IT impact. The second reason is that only these articles focus on IT impact indicating consequences of individual use. Many other IT related articles mention the word impact but did not focus on consequences of individual use. The third reason is that these five articles provided argumentation and motivation as of why the broad range of labels they mentioned can be considered relevant. Many other researches just pick a single or few concepts without providing much explicit motivation. A detailed overview of the literature review is included in appendix F. INTERVIEWING CONTEXT In this section the interviewing context of which the interviews where conducted in are discussed. The goal of this research is to identify concepts of police practices impacted by information technology. For the interviews however, we chose a more specific context. This specific context of the interview is the introduction of an IS called OPP E&S for execution related tasks. We will provide a motivation as why we have chosen this context, details about the information system that will be introduced and the police practices affected by the tasks. Motivation We chose to narrow down the context of interviews towards the topic of a new IS called OPP E&S due to the following reasons: •
Relevancy of the topic for the police force. OPP E&S is a major IS currently under development and the police force is interested in the perspectives of end-­‐users. •
Having a central topic provides a consistent reference point for interviewees to derive examples and narratives from. Generalizability of interviewing context Narrowing down the interviewing context to a more specific context, which is the implementation of OPP E&S, provided the advantage of having a specific topic of discussion during interviews for Measuring the impact of information technology systems on police practices 37 interviewees to base their narratives of. But narrowing down the interviewing context to a more specific context can also result in narratives that only focus on a subsystem of police practices instead of the whole police practices system as a whole. This could compromise the generalizability of retrieved insights and ultimately incomplete identification of concepts to measure. This research will compensate for this by conducting the sorting session data collection using a much wider sample that will not be limited to OPP E&S end-­‐users. Sample We interviewed a total amount of ten end-­‐users. The informants we interviewed were either police officers or administrative workers. The police officers we spoke to were mainly coordinating or supervising officers. The reasoning for this is that in the current E&S context, only coordinators or supervisors are directly involved with working with the information systems. Because of their close interaction with the current systems, these actors are able to provide much more detailed examples and narratives compared to operational officers. We did manage to interview one operational officer, who provided information about IT on a generic level. The administrative workers we spoke to fulfilled roles dependent on their positioning. Administrative workers assigned to support supervising officers naturally conducted tasks similar to what supervising officers would do. Administrative workers assigned to local stations would naturally assist in all the administrative work that operational officers would do. We conducted one interview with a coordinator from Amsterdam; this interview was not used to identify impact constructs. It was rather used to provide the author of this research insight in the working processes. This enabled the author of this research to engage interviews with some basic knowledge about the E&S context. Observational interview sample Rotterdam 1 supervisor, 1 coordinating officer, 1 operational officer Den Hague 3 central coordinating officers, 2 central administrative workers, 2 local administrative workers Amsterdam Central E&S coordinator Table 4.1-­‐2 Sampling for interviews The information system The central topic of the interviewing context is the introduction of OPP E&S. OPP E&S is the first IS to be developed of several IS comprising OPP as a whole. OPP E&S will replace two old systems that are currently used by the police force to manage information regarding so-­‐called execution tasks. The older systems were only built with the sole purpose of retrieving information. OPP E&S will take over this basic functionality but will also have additional functionalities that extend beyond the output of information. These additional functionalities will be implemented with the main focus on mobile accessibility. How the feature of mobile accessibility is expected to affect police practices is explained below. Police practices impacted OPP E&S will be developed and implemented with the main goal of supporting so called E&S related tasks. E&S consists of all operational tasks involved with the identification and detection (signalering) of individuals and the execution of relevant tasks on these identified individuals, such as conducting arrests for detention, seizure of vehicles or collection of fines. 38 Measuring the impact of information technology systems on police practices The current systems only facilitate the input and output of information (figure 4.1-­‐2 left). The process with the current system works as follows: •
•
•
Since the output of the systems still contains raw data, these execution tasks need to be (1) sorted and (2) distributed among the various regional offices and operational officers. This process of sorting and distribution varies greatly among the different former districts in Netherlands. In some districts, the execution tasks are still sorted manually and physically distributed. In other districts, the sorting and distribution is done digitally. Generally, both administrative workers and supervising officers cooperate in the sorting and distributing process. Next to sorting and distributing, supervising officers also (3) manage operative officers (rules and codes of conduct, who does what, how to operate etc.) and control the completed execution tasks conducted by the operative officers. In some districts, administrative workers assist these supervisors in the roll of management and control. If execution tasks are conducted correctly, administrative workers or supervisors (4) input the results back into the systems. Note that in the current system, operative officers are not directly involved with the system. In the current system, the information outflow is indirectly through the ‘filter’ of administrative workers and supervisors. Subsequently, the inflow of information back to the system also goes through administrative workers and supervisors. With the introduction of the new system this information flow will dramatically change (figure 4.1-­‐2 right). With the focus on mobility, the new system will be able to (1) directly send (sorted) execution tasks to the mobile devices of operational officers. Subsequently, operational officers will also be able to directly provide direct (2) input back to the system. Current situation
New situation
Management
Input in system
3
Execution
?
?
1
2
System
4
?
3
1
System
Execution
2
Input in system
Control
Figure 4.1-­‐1 Illustration of the current and future situation regarding the implementation of OPP E&S Many of the issues, concerns and claims expressed by the end-­‐users interviewed are based on current system-­‐related issues but also on concerns related to this new notion of mobility that the new system will introduce. The insights derived from the issues, concerns and claims expressed are described in the following section. For a detailed overview of the documented issues, concerns and claims please consult appendix A. Measuring the impact of information technology systems on police practices 39 To give a very brief and generic description of the issues and concerns: The introduction of the new system creates an environment where operational officers will get more things on their plate. They receive (more?) information directly and also have to put information back in the system by themselves. Officers are worried that this results in more administrative work. On the other hand, supervisors and administrative workers are worried about this uncontrolled flow of information without their consent. INSIGHTS FROM INTERVIEWS CONSIDERING RELEVANT SUBJECTS FOR M EASUREMENT For readability purposes, the insights derived from the interviews are discussed in grouped sections of six. All the narratives provided below are claimed advantages, current issues or future concerns, and therefore considered relevant for measurement (according the interviewees). Preliminary conditions Information technology should just work. If information technology does not work, officers cannot use the information from these systems to do their job. But in the current situation, current information technology doesn’t just work. The current systems used for execution tasks are old and outdated, utilizing an old keyboard based GUI. Many keyboard shortcuts are necessary in order to process information in these systems, making these system very user unfriendly without having significant expertise of how to use these systems. Execution related information is also stored on two different databases. Because of this, many of the information related to execution tasks have to be consulted or entered twice. The preliminary conditions necessary for information technology to just work or not yet met. Therefore, end-­‐users expressed that these preliminary conditions should be addressed when introducing new information technology. In order for information technology to just work, newly introduced systems should be: •
•
•
•
Easy to use. Not compromise time spend on using these systems and performing tasks. Right now, current systems are very cumbersome resulting in tasks taking much longer than necessary. Used at all. If no one uses the system, the system is useless. If only a few officers use the system, the information within the system will be incomplete. Therefore, it is important that everyone fully utilizes the system. Accompanied by proper support. Much to often, end-­‐users only receive an email notifying the introduction of a new system and a thick manual with a lot of errors in it. Entry and reception of information The basic functionalities of information systems are inputting and outputting information. While these basic functionalities are present in the systems, how these systems handle information and assists users in their police practices is still an issue. The information provided by these systems is often polluted, incomplete or not available, which also has consequences for officers doing their job properly. The issues raised during the interviews relate to the following: 40 Measuring the impact of information technology systems on police practices •
Receive accurate information. Current information provided by the systems is polluted and outdated. We can’t rely on the information provided by the current systems and always have to double check. It often happens that we arrest a civilian because the system states that this individual still has unpaid fines, while this individual has already paid for it a long time ago. It becomes problematic, if a system falsely states and individual to be armed and dangerous. •
Receive timely information. While the operation of execution tasks is not a matter of seconds, timing of information received is still important. In the current situation, the reception of information is extremely slow. This is because the flow of information still contains a lot of physical flows. Execution tasks have to printed, sorted and physically distributed. By the time this execution task reaches an officers desk, it is well possible that the concerned individual already paid for the fine, or that the amount of the fine has already been raised. This means that the information on this printed execution form is already outdated and too late. •
Receive complete information. The current information systems provide very incomplete pieces of information. For an officer to get complete information about a case or individual, he/she has to manually retrieve all the information necessary from multiple systems. This problem has been alleviated by the new BVI-­‐IB (which automatically searches multiple databases), but that system has the problem of being very slow. Information is ‘as accurate, (timely and complete) as what’s put into it’(Chan et al., 2001). This notion also holds true for current police systems. Many of the faults and errors in current systems are a consequence of the information input. These input errors cannot solely be contributed to the information technology, because user generated faults and errors play an important role in causing this as well. For now, the social cause of input error will be neglected; instead focus will be set on the technological part that contributes to these issues. The issues relating to information technology in the assistance of inputting information is as follows: •
Input accurate information. Errors and faults cannot solely be contributed to officers being lazy or careless when filling in reports and systems. There are instances where the system forces you to provide inaccurate information. For instance in BVH, the correct form to fill is too cumbersome and complicated, making it tempting for an officer to choose another much simpler form instead. Another example is when the system detects when you fill in a name identical to an existing name. The system forces the latest user to confirm whether both individuals are indeed the same person without having any possibilities to contact the officer that entered the information about the other person. •
Input timely information. Newly gained information is almost never immediately processed into the system. New information often comprises a large amount, has to be documented in a certain way or simply because physical limitations, where a computer is only accessible at a certain location. Officers process this information into the system on their off-­‐patrol days or otherwise are assisted by administrative workers that assist in processing this information in the system for them. Interviewees claim that smart-­‐forms, auto-­‐suggestion and mobile input devices can alleviate some of this lag in inputting information by giving officers the possibility to input essential aspects of certain forms to be inputted immediately. Measuring the impact of information technology systems on police practices 41 •
Input complete information. The inability to input essential information also compromises the completeness of entered information. The time gap between observation and documentation can result in officers simply forgetting certain details. Entry of incomplete information is also caused due to the cumbersome and unclear entry processes or the necessity to enter specific parts of information in different databases. Users Processing of information Information systems feed information that needs to be processed first by users before they are able to apply it to their jobs. This processing of information ranges from simply reading the lines of information of the display but also understanding the lines of information that is fed by the systems. In the current situation, administrative workers and supervisors do most of this information processing. They retrieve the information from the systems and process/filter these pieces of information and translate them into pieces of tasks and additional instructions for operational officers to conduct. As discussed in the previous section, the new system will be able to divert information flow directly to operative officers. With the current system, there are simply physical limitations to direct information directly to operative officers. The new system will have also include the introduction of mobile devices that operative officers can carry, which is the main feature that will enable direct reception and entry of information. This means that dependent on the implementation of this mobile feature, information that flows directly to operative officers will to some extent bypass administrative workers and supervisors. While this feature will have advantages such as faster and more (unfiltered) reception and input of information (previous bullet) and other potential information processing benefits discussed below, there are also concerns expressed by supervisors and administrative workers considering the potential disadvantages. Both aspects of information processing will be discussed below. The claimed advantages that have been acknowledged by interviewees are based on the notion of having more unfiltered and directly available information straight to the operative officers. The subjects claimed by interviewees are as following. •
Providing alternatives to execute tasks. More information that is directly available to operational officers can simply benefit officers by providing more alternatives to choose from in doing their job. To illustrate, if an officer detects a certain individual not opening the door, he will just go back to the office and report the task as not executed. If the same officer would have more information, like car registration information, he would be able to scan the environment for this specific car and verify whether the individual is really not at home or rather just ignoring the door bell. In such a case, an officer would have an alternative to completing the execution task, by confiscating the car instead. •
Assists in recalling information. Interviewees acknowledge that receiving more information directly will assist in recall of information. To illustrate, it takes a long time, from months up to years before an issued fine comes back as an execution task in to the system. It is therefore difficult to recall information surrounding the issued fine, the related individual or whoever issued the fine when this fine comes back in the system as an execution task. Interviewees see potential benefits when a system is able to automatically provide additional information from the time a fine was issued (which officer issued the fine?) for an execution task. 42 Measuring the impact of information technology systems on police practices •
Become more aware of their situation or operational context of task. Interviewees acknowledge that more contextual information about a subject will provide officers on operative duty with better awareness of their surroundings, situation or task contexts. The concerned disadvantages, particularly expressed by supervisors and administrative officers are also based on the same notion of having more unfiltered and directly available information straight to the operative officers. The following concerns were expressed: •
Readability of received information. Supervisors and administrative workers are concerned that operational officers will receive a lot of information on such small mobile screens, thus decreasing the readability of information when compared to paper printouts. •
Understandability of received information. Supervisors and administrative workers are concerned that operational officers will receive unfiltered and unprocessed information with a lot of system jargon (abbreviations and codes) with nobody to provide explanatory notes that would otherwise explain these difficult lines of information. •
Information overload. Directly receiving information could also mean that operational officers will receive an overload of information with both necessary and unnecessary pieces of information, that otherwise could have been filtered out by supervisors and administrative workers with the old situation. Application of information Information technology does not directly lead to users doing a better job, whatever this job may be. Doing a better job is achieved indirectly, by users applying the information they receive from these systems. Instead of explicitly attempting to define their job, interviewees rather expressed various topics that are important aspects that are part of their job and of which concerns exists of how information technology will impact these important aspects of their job. According to the interviewees there are a couple of topics relevant where information technology should assist in within the subject of applying information to do a better job. Many of these subjects are expressed in the form of concerns implied from the notion of how the new system will become. •
Management and control. Better police practices are not just a matter of giving officers more and better information. Officers need to be managed and controlled by their supervisors. Therefore, information technology should also assists supervisors in managing and controlling their team. Supervisors stated their concerns about the new system forgetting the development of features that would assists supervisors in tasks related to management and control. These concerns are amplified by the notion of having direct information flow from systems to mobile devices of operative officers. •
Prioritizing tasks. Doing a better job at policing is not a matter of just doing things or producing things. In policing it is important to prioritize. In some circumstances, officers have to let go of pursuing execution tasks in favor of patrol duty for burglary. Particularly supervisors have expressed concerns surrounding prioritization issues. These concerns are related to the direct information flow from systems to officer mobile devices. Supervisors are concerned that officers will skip tasks prioritized by their supervisors in favor for tasks that show up on their mobile devices. Measuring the impact of information technology systems on police practices 43 •
Innovative behavior. Interviewees stated that new information technology is often inherently coupled with more restriction, which is a matter of concern. The introduction of BVH necessitated users to document reports a certain way and the introduction of summit replaced the freedom of documentation that users previously had with Microsoft word. There are concerns that OPP E&S will restrict the professional freedom that users, both supervisors and operational officers will have when conducting and managing execution tasks. While in most cases, a detailed and restrictive protocol would result in consistency, tailor made decisions would in some circumstances be better. •
Accuracy of decisions. When it comes to policing, officers have to make the right decisions. With the direct feed of unfiltered information, officers having the ability to directly provide input on these devices and the concerns of missing management features, that ultimately operative officers will make more mistakes and errors in fulfilling execution tasks. •
Knowledge management. With the prospects of direct information flows between system and officers, administrative workers and supervisors are aware that some tasks that they do now will become obsolete. This will also mean that some administrative positions will become obsolete as well. A concern expressed by administrative workers and supervisors is that removal of some administrative workers will also entail the removal of specialized knowledge of execution related tasks. This specialized knowledge about execution tasks is especially relevant since police officers do not get any execution-­‐related education during their time at the police academy. Therefore, operative officers currently rely heavily on the knowledge that administrative workers have. There are concerns that this specialized knowledge will disappear among with the displacement of administrative positions. •
Learning. One administrative worker stated that current information technology within the police force has never been used to educate or assist officers in learning new things. He mentioned that there is a lot of potential, especially when considering the new mobile devices, when these systems will include smart tutorials throughout the mobile application. •
Administrative work. Mostly expressed by operative officers is the inherent burden that comes with new information technology. Operative officers expressed their concerns that new features will inherently also introduce more things to do related to filling in forms and reports. This administrative work would eventually decrease the amount of essential time spent on the field. Interaction and service towards the public Part of technological change has implication towards how interaction among officers and towards citizens is impacted by information technology. •
Safety of officers and risk of information loss. Because current blackberry devices are considered difficult, slow and cumbersome, officers are skeptical about having increased reliance on such devices. Officers are worried that by having to rely on such devices during time sensitive or urgent matters, their safety could be compromised. Officers also expressed concerns considering that more information will be accessible on such devices. Careless use or theft could thereby result in a much greater risk of valuable or classified information leaking to the streets. •
Integrity of officers. With prospects of operative officers having much more direct access to information on their mobile devices and reduced layers of control and supervision, a few 44 Measuring the impact of information technology systems on police practices interviewees noted that integrity could become an issue. Officers would be able to use these mobile devices outside of duty to look up criminal records of others. Dependent on the exact implementation, supervisors also raised concerns about purposefully inputting incorrect information into the system without consent of their supervisor. •
Communication and personal approach. Communication among officers and citizens is of utmost importance within the police force. Information technology is often accompanied with digitization of communication streams. While some interviewees believe that digitization only has benefits, some interviewees argue that quality and frequency of communication streams can be diminished. An administrative worker noted that criminality occurs most often in neighborhoods with low level education, poor language skills and low level access to computers. The increased reliance on digital information streams caused by information technology severely diminishes the crucial information stream we receive from these type of citizens. Some interviewees also noted that digitized information flow also makes communication much less personal. When communicating towards colleagues, a personal approach is more likely to get things done. Supervisors noted that they have to personally visit an office or call officers individually to get things done. Emailing them would less likely have the same effect, since officers are spammed with tons of emails daily. •
Professionalism. Following from the skepticism surrounding the use of these mobile devices, some officers are concerned about how they represent themselves towards citizens when using such devices. An example would be the presentation of certain documents where officers would have to struggle through such a device first before being able to show citizens such documents. •
Consistency. As a police organization it is important to be consistent towards citizens so they know exactly what to expect. To illustrate, as a consequence of inaccurate or lack information in the systems it often occurs that an officer communicate information wrongly to citizens. Inconsistency manifests in the form of one officer telling a citizen that he has to go to jail while an other officer tells this citizen that there are no consequences. •
Confidence and stress. Police officers always have to make certain decisions in the field whether they have the right information to make a decision or not. By improved information flow through technology, they are able to make informed decisions instead of gut-­‐feeling decisions, which ultimately results in more confident decisions and less stress while making these decisions. •
Cultural conflicts (IT misunderstanding practitioners). Interviewees raised the issue of an increasing gap between developers and IT professionals. This gap of separation between colleagues working on IT and colleagues working on the field results in developed IT being increasingly inconsiderate to the needs of operational workers. This further increased gap eventually results in IT not being aligned on the practical needs of operational workers. Outcome of policing practices Ultimately information technology is there to assist officers in improving the outcome of their jobs. The effects of information technology on the outcome of policing are very intangible and difficult to narrate. Therefore interviewees did not provide explicit examples but occasionally do state the Measuring the impact of information technology systems on police practices 45 essence of what their job entails. These essential outcomes of policing come down to the following subjects •
Provide service to citizens and respond to criminality. Police officers are servants of society, and exist with the main goal to serve citizens and fighting crime. Information technology should always assist in improving these aspects or at least not diminish the police its ability to achieve these aspects. •
Proactivity. Proactivity is better than reactivity. This ranges from preventing individuals from committing crimes to proactivity in conducting simple tasks. Information technology provides information that enables officers to be more aware of their situation and could therefore improve their ability to proactively react based on this additional information. •
Productivity and efficiency. Although policing entails much more than productivity, at the end, officers and especially supervisors are subject to explicit and implicit forms ‘productivity goals’. For execution tasks, supervisors need to complete a certain amount of execution tasks within a certain time frame. Furthermore, this productivity should also be achieved with maximum efficiency, needing the least amount of effort and resources in achieving such results. 4.1.2 IDENTIFIED CONCEPTS Based on the insight derived from interviews, the empirical study resulted in the identification of 42 concepts considered relevant for measuring. As explained in the chapter of system design, the identification of these concepts for measurement relies on the assumption that subjects raised as issues, concerns or claims is worth measuring. An interviewee would not raise such an issue, concern or claim if they didn’t believe that technological change would have no influence on such subjects. The 42 concepts with subsequent definitions have been included in a table (table 5.2-­‐1) divided in six categories (the same categorization from previous section). Due to the large quantity of concepts identified, this qualitative categorization is only meant for the purpose of improving clarity and readability. The 42 identified concepts are presented under the following categories: •
•
•
•
•
•
Concepts involving preliminary conditions Concepts involving the entry and reception of information Concepts involving the processing of information Concepts relating to the application of information Concepts relating to the interaction and service towards the public Concepts relating the outcome of policing practices 46 Measuring the impact of information technology systems on police practices Concepts Definition Concepts involving preliminary conditions Ease of use The degree to which a person believes that using a particular system would be free from effort Acceptance/attitude The degree to which users come to accept and use a technology Frequency of use The amount of times of which a system is used Facilitating conditions The degree to which an individual believes that an organizational and technical infrastructure exists to support use of the system. Concepts involving the entry and reception of information Entry -­‐ Information from The extent that IT impacts the ability to gather more information from citizens citizens Accurate information The extent to which information contains true value free of errors Timely information The extent to which information is on time, not too early but also not too late. Complete information The extent to which information is complete Receive -­‐ Accurate information The extent to which information contains errors or faulty information. Timely information The extent to which information is on time, not too early but also not too late. Complete information The extent to which information is complete Concepts involving the processing of information Information overload The extent to which information overwhelms an individual Understandability The extent to which information is understandable Recall/retain The extent to which IT assist in recall of job related information Alternatives The extetn that IT enables the consideration of multiple alternatives Awareness The extent to which IT assists in gaining insight of the situation and environment surrounding a task Readability The extent that IT enables individuals to read text. Concepts relating to the application of information Management control The extent that the application helps to regulate work processes and performance Prioritization The extent that IT facilitates an individual to prioritize tasks. The extent that IT facilitates an individual to manage prioritizing of tasks of its team. Accuracy of decisions The extent that an individual makes the right decision or without error. Knowledge The influence of IT on the extent of preserving information that is difficult to transfer to another person by means of writing it down or verbalizing it Learning The extent to which IT enables learning of new knowledge. Task innovation The extent to create and try new things that enable individuals to do things that could not be one before The extent that an individual is able to operate with freedom Administrative work Extent that an individual spends on administrative work. Time use The amount of time that is spent on a task or decision. Concepts relating to the interaction with citizens and colleague’s Professionalism Indicate the concept of image that is portrayed towards the public. This includes terms such as professional, authoritarian, credible. Personal approach Citizens: The extent that an individual is able to personally approach citizens Supervisor – staff: The extent that an individual is able to personally approach their team/supervisor Communication The extent to which individuals communicate and exchange information. Confidence The extent that an individual is certain about the task or decision he made. Consistency The extent that operation of variation in a task when repeated Safety The extent that an individuals safety is at risk. Security The risk that information or data can be stolen or lost. Cultural conflicts The extent of division between different groups of workers Stress Self explanatory Responsibility The extent that an individual is held responsible for faults or errors. Integrity The extent to which IT enables an officer to do what’s right Sense of responsibility The extent that an individual feels responsible for completing tasks correctly. Concepts relating the outcome of policing practices Productivity The amount of tasks completed Efficiency The extent to which utilization of a resource is maximized. Response to crime The extent that IT contributes to emotional or physical tension. Pro-­‐active policing Ability to identify problems more quickly and try to address them. Service towards public The extent to which improved service is provided towards external users. Table 4.1-­‐2 Proposed concepts for measurement Measuring the impact of information technology systems on police practices 47 4.2 MENTAL MAP 4.2.1 FINDINGS FROM INTERVIEWS From the ten interviews that were conducted, some notable observations were made. These observations seem to verify the theoretical assumptions made in chapter two considering the concept of information technology impact and police practices and the motivations towards a systems thinking approach. Complexity of describing police practices When interviewees were confronted with the initial question of how IT impacts their work as a police practitioner they often don’t know where to start. An explanation for this is that their work comprises of so many elements. Police practitioners often state that IT is of utmost importance, since it enables them to receive information necessary to fulfill their work. But when asked to describe how this information assists them in fulfilling their work, they start having difficulty in describing ‘their work’. In order to trigger their narratives, it was necessary to introduce the interviewee with some initial generic examples or topics. After providing the initial triggers, interviewees were increasingly more capable in continuing their narratives towards topics relating to the initial triggers that emerge during their narrative. Complexity in simplifying relationships From analyzing and grouping the statements, we were only able to found one subject that had a substantial majority; which was a general consensus on the utmost importance of receiving information. Other than that, no general majority in any other subjects could be found. For all the other concepts identified, it was difficult to find reasons as why these concepts should be removed from the list. This caused the list with identified concepts to be of substantial size, comprising around 42 concepts. Different perspectives It was also noted that different perspectives on various concepts could be found. Every interview has some kind of opinion on the relevancy of a certain concept whether it does comprise their work as a police practitioner or not or whether it is impacted by IT or not. A clear example that we found was of the concept of safety. Most supervisors that were interviewed considered the concept of safety to be irrelevant. Not because safety doesn’t comprise a police practitioners work, but rather because they state that safety is not impacted by IT use. Therefore safety is not a relevant topic. Operational workers that we spoke to however stated the complete opposite by regarding that safety is most definitely being impacted by IT. This duality on relevancy on the topic of safety is remarkable and leads us to believe that consideration of multiple perspectives is crucial. Complexity of technological change and impact of information technology Another pattern that emerged not only from the interviews but also from observations made is that the consequences of IT and police practices comprises of much larger scale then the fact of certain concepts becoming better or worse. The end-­‐users that we spoke to were all part of a certain category of tasks called ‘execution tasks’. For these end-­‐users, the future IS that will be introduced will most definitely have consequences for how their work becomes better or worse. It may be that the IS will enable these end-­‐users have a more user-­‐friendly interface, thus increasing the usability. Or 48 Measuring the impact of information technology systems on police practices it may be that the IS will make other aspects of their work worse. But what we mean by a much larger aspect is the fact that the IS is expected to drastically change working processes for execution tasks. Because a lot of procedures and tasks will become automated, tasks and procedures that are now being conducted manually will become redundant. This could also result in certain administrative workers to become redundant as well. It is important to consider this aspect of technological change when developing a measurement model. Considering the rich context sensitive nature of process change, process change is less suitable to be represented by a quantitative measurement indicator. Due to the nature of this research, we omitted the aspect of process change as a concept in this research. 4.2.2 CONCEPTUAL MENTAL MODEL Based on these findings we constructed the following mental model. The mental map is illustrated in figure 4.2-­‐1 and color-­‐coded based on the categories defined in 5.2 to improve readability. These categorization are roughly based on the levels of information output proposed by (Mason, 1978) being output of information (yellow), processing of information(green), application of information (pink) and outcome of information (red). Based on outcomes of the interviews, additional categories were formulated being preliminary terms (blue), input of information (yellow) and interaction through information (brown). These categorizations were based on strong clustering in the interview narratives and do not imply that these categories can be viewed as isolated subsystems. The mental map contains all the identified concepts mentioned above including relationships between them. Note that the relationships are unverified and invalidated. Therefore these relationships primarily serve as an indication. Further research is necessary to validate and improve the relationships contained in the mental map. The mental map illustrates the structure of police practices. As figure 4.2-­‐1 shows, it can be seen that although not necessarily linear, a process-­‐like structure can be seen starting from preliminary conditions, inputting and receiving information, understanding of information, application of information to outcome of applying information. The figure also shows receiving information to be an important ‘hub’ within the mental map. Measuring the impact of information technology systems on police practices 49 Figure 4.2-­‐1 Mental map of police practices system impacted by information technology 4.3 CHAPTER CONCLUSION In this chapter, the identified concepts from the literature review and interviews are presented. This resulted in a list with a total amount of 42 concepts relevant for measurement. Relevancy of these 42 concepts is based on narratives provided by interviewees that were raised as a current issue or as a future concern. Based on these 42 concepts and insights derived from the interviews, a mental map was constructed. This mental map visualizes how the police practices system looks like from the point of view of this researcher. Although, established relationships between the various concepts in the mental map are incomplete, it still provides a visualization of the complexity of how the police practices landscape looks like. The mental map shows output of information to be a very important hub within the system, but next to this insight the remaining concepts and its relationship show a complex pattern. The next chapter will focus on validation of this mental map, primarily on the relevancy of the concepts included in the mental map. 50 Measuring the impact of information technology systems on police practices 5. VALIDATION The goal of validation is focused on validating the relevancy of concepts included in the mental map. The validation chapter is arranged as follows. The first section will discuss how the results of the card sorting sessions were derived and what these scores tell about the overall relevancy of the identified concepts. The second section will discuss the concepts scoring the lowest on relevancy. The third section will give an overview of all the specific given relevancy scores for each concept, visualized in a heat map. The fourth section discusses the validity of choosing a multi actor perspective. The fifth section discusses the validity of assuming a system thinking approach. 5.1 DATA COLLECTION FOR VALIDATION In order to validate the relevancy of concepts included in the mental map, this research used empirical data derived from card sorting sessions with police practitioners. End-­‐users were asked to sort 42 concepts in nine groups ranging from the highest relevancy and the lowest relevancy. We asked the police practitioners to sort cards based on what they find most important regarding the relationship between information technology and their own personal work experiences. We also explicitly explained that information technology includes all the software and hardware they use in assistance of their work. This research decided upon a 9 most relevant to least relevant scale instead of a binary relevant/irrelevant scale, since a 9 scale provides much more information allowing analysis of relative relevancy. Figure 5.1-­‐1: Introduction letter to personal sorting session The sorting was done in a two-­‐step process, which is to sort in groups of three and again in groups of three, creating a categorization within nine groups. The sorting sessions were conducted face-­‐to-­‐face using two ipads simultaneously. Due to the complexity of the topic, face-­‐to-­‐face sorting in a semi-­‐
interview setting was deliberately chosen. This face to face setting enabled the data collector to explain the card sorting session in an interactive way, giving the respondents the possibility to ask questions and make comments regarding unclarity about the goal of the sorting sessions or the sorting cards. The 42 sorting cards are included in appendix C. 5.1.1 SAMPLING Sorting sessions were held with 37 respondents. Unfortunately, 10 data entries were lost due to loss of data or data corrupting. 1 dataset had to be removed because the respondent sorted the statements incorrectly. This resulted in a total amount of 25 usable data entries. While the interviews used for the identification of concepts were held with a very select sample of users from OPP E&S, the sampling used in the sorting sessions was focused on gaining a much more generic sample of end-­‐
users that do not necessarily focus on the context of OPP E&S. Although four of the 25 respondents were reoccurring respondents from the sample used for the interviews, we believe that this amounts Measuring the impact of information technology systems on police practices 51 to a proportional representation between future OPP E&S users but users of other IT systems. Note that from a geographical perspective, the sample mainly consists of respondents of the randstad area (1 respondent from outside randstad). Gender Location Function Stakeholder type 16% female 84% male 9 Rotterdam 1 breda 15 Den Haag 7 Hoofdagent 1 Agent 2 Administratief 4 ‘Coordinator’ 3 Inspecteur 3 Brigadier 2 HKD 3 Recerche 7 Leidinggevende 18 operationeel 5.1.2 SORTING APPLICATION The app used is called qCard Sorting. The application allows users to sort cards by drag and dropping cards in groups of 3 and again in groups of 3, resulting in card sorted in stacks of 9. The app contains an overview of the cards that are placed in a stack and enables respondents to scroll through the cards and redraging cards to another stack. This provides a visual overview of the cards and encourages relative comparison. The data is written to a csv file and exported to excel and R for data analysis. 52 Measuring the impact of information technology systems on police practices Figure 5.1-­‐2: (a) cardsort app as displayed in the appstore (b) cardsorting view during sorting in groups 1-­‐3,4-­‐6,7-­‐9 (c) final overview of sorted cards after finalized sorting (d) officer during card sorting session on ipad 5.2 VALIDATION OF IDENTIFIED CONCEPTS 25 respondents sorted a total amount of 1050 cards. Out of the 1050 cards that were sorted, over 76% of those cards were sorted in the category medium to high relevancy and over 22% of those cards were sorted in the category 9, making category 9 containing the most cards. The distributions of how these concepts were sorted are illustrated in figure 5.2-­‐1. With a majority of cards sorted in the higher categories, these results seem to provide evidence that a majority of the concepts identified during the interviews can be considered valid. Note that during card sorting sessions, respondents were encouraged to sort concepts to whatever category they liked. We explicitly encouraged respondents to put concepts in the low relevancy category if it didn’t make any sense, still a majority of the cards were sorted in the higher categories. Measuring the impact of information technology systems on police practices 53 9 hi 8 hi 24% 36% High revelance 40% 8% 7% Medium relevance 9% 12% 11% 6 med 11% 13% Low relevance 7 hi 22% 7% 5 med 4 med 3 low 2 low Figure 5.2-­‐1: (1) Distribution of concepts sorted based on high, med and low relevancy (2) Distribution of concepts sorted based on 9-­‐scale rating 5.3 CONCEPTS CONSIDERED TO BE OF LOW RELEVANCE As section 6.1 shows, over 76% of all the cards were sorted in the higher categories regarding rate of relevancy. This section provides insight in the concepts considered to be of low relevancy. A figure was compiled containing all 42 concepts ranked based on highest relevancy and lowest relevancy. The relevancy is based on the average scores given to each concept. The figure also includes medians and boxplots of each concept. The figure is shown in figure 5.3-­‐1. Dependent on what threshold one defines as a minimum score for relevancy; it is possible to identify concepts to be considered irrelevant. If this threshold is set at an average score and median of four, four concepts can be considered the least relevant concepts or even irrelevant. These concepts are integrity, responsibility, personal approach and cultural conflicts. A reoccurring explanation given by respondents as of why these concepts were ranked least relevant is that current or future states of these subjects cannot be attributed to IT, because they are largely determined by how officers themselves handle such situations and how they deal with these issues. Using integrity as an example, an officer provided the following illustration: “With BVI-­‐IB I can now take this blackberry home with me and look up my daughters new boyfriend to see whether he has a record or not. The technology does enable me to compromise my integrity, but it is my full responsibility to preserve this integrity by not doing that”. “If integrity is a matter of a concern, we cannot blame the technology, instead we should blame the level of integrity of the person using that technology”. 54 Measuring the impact of information technology systems on police practices Figure 5.3-­‐1 Average relevancy score for each concept sorted based on highest score relevancy 5.4 CONCEPTS CONSIDERED TO BE OF HIGH RELEVANCE Next to identification of concepts of lowest relevance, it is also interesting to identify the concepts that are of the highest relevancy. The identification of these concepts is useful in serving as indicators to include in the measurement tool. The identification of concepts of highest relevance in this research is selected based on the following assumptions: •
•
Concepts having an average score of 6.5 or higher can be considered the most relevant concepts Both operative and supervising users consider different concepts to be relevant, therefore it is important to identify most relevant concepts of each separate perspective. The identification of most relevant concepts from the scores provided by operational users is based of figure 5.4-­‐1. The table shows the following nine concepts to be most relevant to operational workers: Measuring the impact of information technology systems on police practices 55 Concept Output_accurate_info Output _complete_info Output _timely_info Time use Input_accurate_info Ease of use Administrative_work Knowledge preserve Communication Score 7.8 7.4 7.3 7.3 7.2 7.0 6.8 6.8 6.6 Figure 5.4-­‐1 highest relevancy scores given by operational officers The identification of highest scoring concepts based on the scores provided by supervising users is shown of figure 5.4-­‐2. The table shows seven additional concepts to be relevant from the perspective of supervising users. Concept Productivity Score Acceptance 8.6 8.3 8.1 8.1 8.0 7.9 7.9 7.9 7.7 7.6 7.1 Management control 6.9 Time use Ease of use Efficiency Output_of complete_info Output of_accurate_info Task_innovation Proactivity Input_accurate_info Output_timely_info Figure 5.4-­‐2 highest relevancy scores given by supervisors This identification process of most relevant indicators from the perspective of operational users and supervisors leads to the identification of 15 indicators. These indicators are presented below in separate categories to improve readability. The graphic below shows the following concepts that are considered the highest relevance in the police practices system. For police practitioners in general, the following indicators relating to information technology and police practices are considered most relevant: 56 Measuring the impact of information technology systems on police practices •
•
•
•
•
•
The technology should be easy to use and not time consuming The technology should assist in inputting accurate information The technology should assists in receiving complete, accurate and timely information The technology should assist in improving communication The technology should relieve the amount of administrative work that a user has to conduct The technology should assist in preserving policing knowledge. For supervisors specifically, the following indicators are considered relevant, more then what other types of end-­‐users consider relevant, next to the indicators specified above: •
•
•
•
•
•
Increase acceptance of users in using technology Assisting the user to do perform tasks innovatively Assist in managing and controlling their team Achieve proactivity in conducting tasks Assist in increasing productivity Assist in increasing efficiency. Preliminary
Input
Output
Complete
informatio
n
Ease of
use
Time use
Acceptan
ce
Accurate
informatio
n
Accurate
informatio
n
Timely
informatio
n
Application
Communi
cation
Task
innovatio
n
Adminstr
ative
tasks
Knowledg
e
preservati
on
Outcome
Proactivit
y
Productiv
ity
Managem
ent
control
Efficiency
Figure 5.4-­‐3 Proposed concepts for inclusion in measurement tool 5.5 VALIDATION OF MENTAL MAP The mental map constructed in the previous section has been developed based on insights derived from the interviews mainly being the complexity, non-­‐reductionistic property. For validation purposes, this mental map includes a heat map overlay based on the relevancy scores derived from the sorting sessions. The heat map shows bigger and brighter circles according to higher scores of relevancy. From a validation point of view, the heat map provides some visual confirmation of the validity of the mental map. Although the heat map provides a lot of interesting insights regarding the validity of the mental map, we highlight some of the most interesting ones. •
Outputs considered more relevant than inputs of information The heat map shows that output of information is considered to be more important than input of information. For the input of information, accurate input of information is considered more relevant than complete and timely input of information. This is a notable observation, although output of information is greatly dependent on the input information, respondents consider input of information to be of lesser importance. This observation is consistent with insights in interviews however, as many interviewees noted “the irony of the Measuring the impact of information technology systems on police practices 57 •
•
•
•
situation is that officers want to receive good quality information without wanting to be bothered with inputting good quality information themselves” In the end, it comes down to productivity and efficiency In the end productivity and efficiency were still rated to be of great importance, which is an interesting observation, particularly since in the interviews productivity and efficiency was not a popular topic of narration. Time use and ease of use are important ‘hygiene factors’ These findings are in line with interviews, which indicated that current systems are just not easy to use, and to slow. Administrative work and knowledge preservation considered most important when applying information from IT Out of all of the concepts identified in the category ‘applying information’, administrative work and knowledge preservation are identified as most important. It is notable that these two factors are apparently more important than other concepts relating to the application of information such as learning, prioritization and management. Communication Out of all the concept relating to interaction, communication was considered the most important, close runner ups are safety and risk of information loss. All other concepts such as stress, integrity, confidence, professionally and consistency were considered far less important. 58 Measuring the impact of information technology systems on police practices Figure 5.5-­‐1 Mental map of the police practices system including circle-­‐size based on average relevancy scores 5.6 VALIDATION OF DUAL PERSPECTIVE In order to validate the dual perspective of this research we asked each respondent to make an explicit choice between being an operative or supervising role. Based on the choices respondents provided, we compiled two charts with each containing concepts sorted based on relevancy from both perspective. The chart ranked sorted from the perspective of operative users is included in the appendix. The chart below visualizes the relevancy of the 42 concepts sorted on average relevancy from the perspective of supervisors. As indicated by the grey circle, it clearly shows that there are large gaps between the average relevancy scores given by supervisors and the average relevancy given by operational users. A large amount of concepts were given higher relevancy scores by supervisors when compared to operational users. Man-­‐Whitney u tests were conducted to assess the significance of these differences and the results of the u-­‐test show four of the 42 concepts to have significant differences in the test results (appendix D). This observation confirms the existence of a difference between what supervisors and operative users that should not be ignored. It is important to include this dual perspective in the identification of relevant indicators for inclusion in a measurement tool Measuring the impact of information technology systems on police practices 59 Figure 5.6-­‐1 Average scores of concepts sorted on highest scores + distinction between operative & supervisors 5.7 VALIDATION OF ASSUMPTIONS TO CHOOSE SYSTEMS THINKING APPROACH The results from the sorting sessions in this chapter seem to confirm the validity of choosing a systems thinking approach in this research. The pie chart showed 76% of the concepts to be ranked in medium to high relevancy. Furthermore, the average scores displayed in figure 5.3-­‐1 indicate sixteen concepts to have an average relevancy score above a six and 31 concepts to have an average relevancy score higher than five. It is interesting to witness these results, especially since we encouraged respondents to sort cards however they wanted. We even explicitly stated that respondents were allowed to sort all cords in the lower categories if the cards didn’t make sense. Still, we ended up with more than 31 concepts to have an average score of medium or higher relevancy. These results clearly indicate that the concept of police practice cannot be simplified to a single or even a couple of concepts, thus validating the appropriateness of choosing a system thinking approach. 60 Measuring the impact of information technology systems on police practices 5.8 CONCLUSION In this chapter, the efforts towards validation were discussed. These efforts included the assessment of the overall validity of the concepts, specific validity of the identified concepts, the mental map, the dual perspective and the appropriateness of choosing a system thinking approach. Based on analysis of the card sorting data, it is safe to conclude that generally, a large amount of concepts included in the mental map can be considered as relevant. The concepts considered to be not as relevant have also been discussed. This chapter also validates the relevancy of choosing a dual actor perspective as well as the appropriateness of choosing a system thinking approach. This chapter does not contain validation efforts regarding the validity of relationships within the mental map, as gaining understanding of relationships is not the primary focus of this research. In the next chapter, results from this research will be used to propose considerations and recommendations for designing a measurement tool. Measuring the impact of information technology systems on police practices 61 6. DESIGNING A MEASUREMENT TOOL As discussed in the background section of the introduction, this research was conducted in to assess the police force current impact measurement methods for future IT implementations. Keeping this practical motivation in mind, this chapter explicitly reflects on the practical side of this research. Mainly, how the outcomes of this research can be used to improve the measurement practices used in the current situation. This chapter starts with a brief introduction of the belevingsmonitor, the current measurement tool being used. Afterwards, the goal of measurement is discussed by the approach that will be applied in making recommendations. This chapter follows with considerations for the owner of the belevingsmonitor to consider, explicit recommendations. Finally, based on the considerations and recommendations, this chapter provides four example scenarios of how measurement can be conducted in the future including its implications for decision-­‐making based of the measurement tool. 6.1 THE BELEVINGSMONITOR In 2011, the Dutch national court of audit (Algemene rekenkamer, 2011) published a report regarding the performance of IT facilities within the police force, which concluded that the IT facilities within the police practices has not shown any significant improvements for the past ten years. One of its main findings was the poor user-­‐friendliness and usefulness of systems supporting operational officers. These findings directly led to the establishment of the Aanvalsprogramma politie 2011-­‐2014 (AVP), which consisted of large scale plans to improve and modernize most operational IT facilities within police force. 4
The belevingsmonitor was brought to life alongside AVP as an instrument to monitor and report changes brought forth by specific IS implementations back to the management board of the AVP (Hoefnagel, 2013). The belevingsmonitor adopted the use of technology acceptance models as its main rigor for indicators to include in the belevingsmonitor. The choice for a technology acceptance model is motivated by the AVP of having ‘more user-­‐friendliness and increased usability of IT for police practitioners’ as its main objective. Since technology acceptance models contain ease-­‐of-­‐use and perceived usefulness as its most important ‘predictors’, the adoption of technology acceptance models seems a logical choice as a rigor for the belevingsmonitor. In order to measure the impact of IS projects on individual police officers, belevingsmonitor is an online survey issued to a large amount of police officers. The online survey is based on constructs provided by the Unified Theory of Acceptance and Use of Technology (UTAUT) (Venkatesh, Morris, 4
The belevingsmonitor is divided in three sub studies with each their specific goals. The first sub study is a half-­‐yearly measurement that focuses on measuring the overall impact of ICT on police officers, the second sub study is a yearly study focused on exploration and generation of relevant themes related to IS experiences of police officers. The third sub study is focused on measuring the impact of separate IS implementation projects, such as E&S, on individual police officers. Altough the belevingsmonitor contains three substudies, this research only refers to the third substudy. Therefore references to the term ‘belevingsmonitor’ actually entails references to ‘belevingsmonitor substudy three’. 62 Measuring the impact of information technology systems on police practices Davis, & Davis, 2003). These constructs are performance expectancy, effort expectancy, social influence and facilitating conditions. Thus the belevingsmonitor contains six indicators each being represented by questionnaire items as depicted in table 6.1-­‐1. Since the police force already has an existing measurement method, this research will not propose operationalization of measurement from scratch. Rather, this research will critically reflect upon the current state of the belevingsmonitor and make recommendations for possible adjustments to the belevingsmonitor. Before making these recommendations, this section discusses the purpose and approach of how these recommendations will be made. Concept Gebruiksgemak (Effort expectancy) Definition De mate waarin een persoon gelooft dat het gebruik van een technologie vrij is van inspanning Bruikbaarheid (performance expectancy) De mate waarin een persoon gelooft dat het gebruik van een technologie zijn prestaties zal verbeteren op het werk Sociale invloed (social influence) De mate waarin een individu waarneemt dat belangrijke personen uit de omgeving vinden dat hij of zij het nieuwe system zou moeten gebruiken Faciliterende omstandigheden (facilitating conditions) Acceptatie (intention to use) Ervaring met gerelateerde techniek De mate waarin een individu gelooft dat er een organisatorische -­‐ en technische infrastructuur bestaat die gebruik van het systeem ondersteunt Een subjectieve waarschijnlijkheid van gedrag De mate waarin een individu gelooft ervaring te hebben met gerelateerde technologie Table 6.1-­‐1: Current survey design of belevingsmonitor Item (in vragenlijst) Mijn interactie met SUMMIT is helder en begrijpelijk Interactie met SUMMIT kost niet veel mentale inspanning Ik vind SUMMIT gemakkelijk om te gebruiken Ik vind het gemakkelijk om SUMMIT te laten doen wat ik wil Gebruik van SUMMIT verbeterd mijn prestaties op het werk Gebruik van SUMMIT verbeterd mijn productiviteit in mijn werk Gebruik van SUMMIT vergroot mijn effectiviteit in mijn werk Ik vind SUMMIT nuttig voor mijn werk Mijn directe collega’s vinden dat ik SUMMIT moet gebruiken In het algemeen, heeft de organisatie het gebruik van SUMMIT gestimuleerd Mijn leidinggevende heeft gebruik van SUMMIT gestimuleerd Ik heb de hulpbronnen die nodig zijn om SUMMIT te gebruiken Ik heb de kennis om SUMMIT te kunnen gebruiken SUMMIT is compatible met andere systemen die ik gebruik Ik kan met iemand contact opnemen voor ondersteuning en problemen met SUMMIT. Ik zal SUMMIT zo vaak mogelijk blijven gebruiken Ik voorspel dat ik SUMMIT zo vaak mogelijk gebruik Ik ben van plan SUMMIT zo vaak mogelijk te gebruiken 19. Ik heb ervaring met het gebruik van informatiesystemen 6.2 REFLECTING THE GOAL OF THE BELEVINGSMONITOR As stated in the Belevingsmonitor proposal report, the measurement goal of the belevingsmonitor is “To assess the impact of individual subprojects of the AVP in light of main objectives of the AVP being: more ease of use and usefulness of police IT facilities for police practitioners” (Hoefnagel, 2013). Although not explicitly stated, this measurement goal is specifically aimed at assessing the impact of individual IS projects on the perceived ease of use and usefulness of police IT. This particular focus on perceptions follows from the terminology used for in the belevingsmonitor, containing the word ‘beleving’ literally translating to perceptions. The belevingsmonitor operationalizes measurement of perceived ease of use and usefulness by issuing online surveys containing 19 items before and after implementations of certain IS projects (table 6.1-­‐1). But why exactly do we measure the goal stated above? “If measurement is simply viewed as a data-­‐
collection and reporting exercise, it will serve little purpose to a community. Ultimately collection of data is not just for the sake of reporting, but also to use this knowledge to manage, adjust and implement improvements” (ICMA (2000). In other words, the belevingsmonitor’s goal is not just to measure perceived ease of use and usefulness for the sake of measuring it, but to use this knowledge Measuring the impact of information technology systems on police practices 63 in assisting the decisions that decision makers have to make in IT implementations and improvements. Assuming that measurement is meant to provide knowledge to manage, adjust and implement improvements, we will make recommendations based off this notion. 6.2.1 APPROACH According to contingency theory, there is no one right way in doing things. Every decision has its advantages and disadvantages. Therefore the way something should be implemented is dependent on a decision makers decision style is and what a decision maker considers most relevant in specific situations. This also means that how the belevingsmonitor should and must be adjusted for future measurements ultimately remain a responsibility of the owner of the belevingsmonitor. Assuming contingency, this research will not recommend the most optimal way in improving the belevingsmonitor, since such single one size fits all recommendation does not exist. Instead of proposing one single solution towards improvement of the belevingsmonitor, this research will propose considerations on the following elements of how the current belevingsmonitor is designed: •
•
•
Choosing the right indicators (why focus on ease of use and usefulness?) Collecting data (why focus on ‘beleving’ measured through surveys?) Possibility to conduct measurement before and after implementations Following from these critical considerations while also making recommendations, this research will provide four possible future scenarios regarding how the belevingsmonitor can be continued including its implications for effective decision-­‐making manage for IT improvements. Definition of effective decision-­‐making Decision-­‐making is effective if decision-­‐makers are able to make informed decisions based on knowledge on information that matter the most and can be relied upon, while taxing the least amount of resources to make those informed decisions. Recommendations in this chapter should respect the following criteria: •
•
•
Data collection should collect as much as knowledge (in the current situation, perceptions) as efficiently as possible while considering: Minimal taxation on frontline police-­‐practitioners Minimal taxation on the resources of the data collector himself or herself 6.3 CONSIDERATIONS AND RECOMMENDATIONS This section discusses considerations regarding three key elements of the belevingsmonitor and also follows up with recommendations. Section 7.3.1 discusses the indicators to include in the belevingsmonitor. Section 7.3.2 discusses the data collection method for the belevingsmonitor. Section 7.3.3 discusses the possibility of before and after comparisons. Section 7.3.4 provides four possible future measurement scenarios based on the considerations and recommendations discussed in section 7.3.1 through 7.3.3. 6.3.1
INDICATORS TO INCLUDE IN THE BELEVINGSMONITOR The belevingsmonitor aligned its measurement indicators directly to the main objectives of the AVP, which are ease of use and usefulness. Building forth on these two main indicators, the 64 Measuring the impact of information technology systems on police practices belevingsmonitor adopted UTAUT, which added four additional concepts to these two indicators. This means that in the current belevingsmonitor six key concepts are used as indicators relevant for measurement in the belevingsmonitor. In this research an exploration study was conducted towards the various concepts relevant for measurement in order to gain insight in the impact of IT on police practices. As results from the interviews show, 42 concepts relating to information technology and police practices have been found to be relevant. In order to consider recommendations towards the belevingsmonitor, a table has been compiled containing the 42 concepts identified in this research sorted based on the relevancy scores. The table also contains a direct reflection whether the identified concepts are represented in the current belevingsmonitor. The table is presented in table 6.3-­‐1. Relevancy score 7.884615385 7.56 7.5 7.423076923 7.32 7.307692308 7.08 Concepts identified in this research Output accurate information Time Use Output complete information Input accurate information Ease of use Output timely information Productivity Included in the belevingsmonitor? Yes, 4 survey items Yes, 1 directly related survey item and 3 relating survey items 6.84 Administrative work 6.76 Knowledge preservation 6.72 Efficiency 6.68 Proactivity 6.461538462 Input complete information 6.4 Communication 6.269230769 Input timely information 6.16 Readability 6.04 Response to crime 5.88 Accuracy of decisions 5.88 Frequency of use 5.846153846 Recall 5.84 Task innovation 5.64 Risk of information loss 5.52 Safety 5.32 Acceptance Yes 5.32 Information from citizens 5.269230769 Information overload 5.2 Learning 5.16 Facilitarity conditions Yes 5.153846154 Understanding 5.12 Alternatives 5 Management control 5 Realization 4.84 Consistency 4.76 Professionalism 4.68 Service towards citizens 4.32 Confidence 4.32 Prioritization 4.04 Stress 3.92 Situational awareness 3.6 Responsibility 3.6 Integrity 3.52 Personal approach 3.24 Cultural conflicts Concepts included in the belevingsmonitor which are not identified in this research Experience Social influence Table 6.3-­‐1: Comparison table concepts from this research vs. concepts included in the belevingsmonitor Measuring the impact of information technology systems on police practices 65 Following the rule of contingency, there is no right or wrong set of indicators for measurement. Thus, the concepts identified in this research are not necessarily the right set of indicators while the concepts used as indicators in the belevingsmonitor are also not necessarily wrong. However, based on the table above, this research proposes the owner of the belevingsmonitor to critically reflect the indicators currently included in the belevingsmonitor. The owner of the belevingsmonitor is advised to consider adjustment of indicators included in the belevingsmonitor based on what the goal of the belevingsmonitor is. Since the goal of the belevingsmonitor is not to understand and predict acceptance, it is advised to alter some indicators included in the belevingsmonitor. Ideally, it would be recommended to include all 42 concepts as identified in the table, but since this research consideres resource efficiency as a criteria ( as discussed in 7.3.1), this research takes into consideration that not all 42 concepts can be included in the belevingsmonitor. Instead, this research recommends the following alterations to the indicators included in the belevingsmonitor. RECOMMENDATIONS By directly reflecting the results of this study next to the indicators included in the belevingsmonitor, the following insights can be derived: •
•
•
•
Keep of ease-­‐of-­‐use as an indicator: Ease-­‐of-­‐use remains an important indicator for IT impact on police practices. Due to the high relevancy score, ease of use should remain in the belevingsmonitor. Although the relevancy of ease-­‐of-­‐use remains important, the table 6.3-­‐1 indicates four other concepts that are considered far more important than ease-­‐of-­‐use. The concepts considered more relevant are the output of accurate information, time use, output of complete information and input of accurate information. Include aspects of information quality as indicators: One of the main findings of this research, confirmed by the highest relevancy scores is that output of information is of utmost importance. Concepts relating to information quality should therefore be included as indicators in the belevingsmonitor. These indicators are output and input of complete, accurate and timely information (six indicators). Include time use: Time use of information systems is regarded the second most important subject regarding technology and police practices and should therefore be included as indicator in the belevingsmonitor. Replace usability items with less generic indicators: The current belevingsmonitor uses four survey items to represent the concept usability (depicted in textbox below). Results from this research however indicates that usability can be made much more explicit than the current terms included in the survey items representing usability. Interviewees themselves never referred to the term usable, but rather narrated topics much more explicit than the concept of being usable. For example, IT can be usable in terms of alleviating the amount of administrative work they have to do, IT can be usable in improving communication among practitioners, IT can be usable in terms of assisting management and control between an individual and their team etc. It is difficult to draw an exact line as of which concepts of the total amount of 42 concepts can be allocated to represent ‘usability’, but the 42 concepts identified do show that usability can be made far less generic than the four survey items currently included in the belevingsmonitor. 66 Measuring the impact of information technology systems on police practices Usability of technology in the Survey items used to represent the concept ‘usability’: context of policing can be replaced with much more Gebruik van SUMMIT verbeterd mijn prestaties op het werk Gebruik van SUMMIT verbeterd mijn productiviteit in mijn specific concepts. For werk illustration purposes, high Gebruik van SUMMIT vergroot mijn effectiviteit in mijn werk scoring candidates eligible for Ik vind SUMMIT nuttig voor mijn werk replacement of the four current items representing usability have been indicated in blue and exemplified in survey items in the table 6.3-­‐2. Survey items used to represent the concept ‘usability’: Gebruik van SUMMIT ondersteunt mij om productiever te zijn in het afronden van taken Gebruik van SUMMIT vermindert mijn administratieve lasten Gebruik van SUMMIT ondersteunt mij in het bewaren en behouden van politie kennis binnen de organisatie Gebruik van SUMMIT ondersteunt mij in het communiceren met mijn team en burgers. Gebruik van SUMMIT ondersteunt mij in om proactiever om te gaan in het initiëren van algemene taken en politiewerk Table 6.3-­‐2 Survey item examples for 'usability' Measuring the impact of information technology systems on police practices 67 •
Relevancy of facilitating conditions as an indicator: Results of the interviews indicated measurement of facilitating conditions to be a relevant indicator, since improper facilitating conditions interferes with proper and effective use of policing IT. Results of the sorting sessions however indicated facilitating conditions to be th
ranked 27 out of the 42 concepts considered relevant. Based on this knowledge, this research recommends the owner of the belevingsmonitor to critically reflect upon the relevancy of facilitating as an indicator to include in the belevingsmonitor. Particularly since 26 other much more relevant concepts are not included in the current form of the belevingsmonitor, facilitating conditions as an indicator can be replaced by far more relevant indicators instead. •
Relevancy of acceptance as an indicator: Results of the interviews indicated measurement of acceptance to be a relevant indicator; particularly supervising interviewees noted that they find it important that their subordinates are not only using the systems because they have to, but also because they are willing to use the system. If subordinates use the system because they want it themselves, supervisors can spend less effort on controlling and enforcing their subordinates whether they are really using the systems. The relevancy of acceptance is also reflected in the ratings. th
Although general relevancy scores indicated acceptance to have a score of 23 out of the 42, th
supervisors ranked acceptance 11 out of the 42 concepts. •
Relevancy of social influence as an indicator: The relevancy of social influence was not confirmed in the interviews. A logical explanation follows from the fact that social influence tells something about the degree of which an individual is susceptible to social pressure. The susceptibility to social influence is determined by ones own personality, norms and values (Kinzig et al., 2013). Any sort of technology is less likely to impact an individual’s personality, norms and values. Measuring social influence is particularly relevant in predicting and understanding system use, since system usage can be prone to social pressure. In terms of technological impact and police practices however, social influence is a less logical concept relevant for measurement. Since the goal of the belevingsmonitor is not to predict acceptance, rather mapping the impact of IS implementations on police practitioners, the owner of the belevingsmonitor is advised to reflect on omitting inclusion of social influence as a indicator in the belevingsmonitor. Relevancy of experience as an indicator: Following from the interviewees, no mentioning of topics relating to experience has been discussed. A logical explanation for the absence of experience is that experience comes with time spent on IT. Experience does not improve or negate if additional IT is implemented. Thus, in terms of technological impact and police practices it seems strange to be measuring experience, since experience is not subject to change because of IT. Therefore, the owner of the belevingsmonitor is advised to reflect on omitting inclusion of experience as a indicator in the belevingsmonitor. If the owner expects levels of experience to have significant differences among users of different systems, the owner should maintain inclusion of experience as an indicator. •
68 Measuring the impact of information technology systems on police practices 6.3.2
DATA COLLECTION IN THE BELEVINGSMONITOR ‘You can’t manage what you can’t measure’. Thus, data is necessary to gain the knowledge necessary to make informed decision. With a focus on ‘beleving’, the belevingsmonitor utilizes surveys. Surveys are a very effective way in obtaining a large amount of perceptions from different end-­‐users. The belevingsmonitor in its current design consist of a 19-­‐item survey issued online. This means that each 5
survey can be completed in approximately four minutes . Having this short survey length is ideal since it requires minimum taxation of end-­‐users their time. Additionally, online administration of these surveys also adds that minimum effort is required from the data collector himself or herself in collecting perceptions from a very large amount of end-­‐users at once. In this research, 42 indicators were identified to be relevant and thus to be potential candidates for inclusion in the belevingsmonitor. Considering that data collection requires resources, it would be unpractical to recommend the belevingsmonitor to collect additional data on all 42 indicators identified in this research. However, more knowledge for informed decisions is still a crucial criteria, thus ignoring these identified indicators would be also be an unwise decisions. Therefore, consideration of other methods of data collection becomes increasingly relevant when the owner of the belevingsmonitor considers to extend the belevingsmonitor with more indicators without increasing the taxation on end-­‐users or data collectors. Considering that data collection should provide the broadest knowledge will minimizing taxation of resources, this research suggests other methods of data collection besides the use of surveys. Why do we have to limit ourselves to surveys if other sources of information are readily available? Therefore, from a data collection perspective this research would like to propose other methods of data collection supplementing the current design of the belevingsmonitor. •
•
Unobtrusive (digital) records Unobtrusive records are methods for collection of data that do not require the cooperation of the subjects and, in fact, may be “invisible” to them (Fairfax County, 2007). This measure enables gathering data without interfering in the ongoing flow of events or the observed individual to influence the outcome. The digital aspect of this measure entails that once the mechanics are installed or digitally programmed, the instrument collects information automatically. This results in minimal effort of data collection for the data collector. This type of measure is suited for collecting so called “hard data,” and is most frequently used to describe the measures such as inputs (staff, budget dollars); outputs (clients served); and efficiency (cost per client). Quantitative data for the purpose of data collection is simply how many -­‐ how many clients, how many purchase orders, how many medical exams, etc. This type of data can often be collected on simple check sheets (manual) or through automated means such as spreadsheets, databases, or mainframe applications. Especially with the advent of OPP E&S and other future IS implementation many of the police practices that are currently conducted or recorded manually will become either automated or digitalized. Either intentional or non intentional, this means that a lot of information will be captured digitally and therefore be readily available. With this increase of readily available digital data, it would be interesting to embrace this new source of information in the belevingsmonitor. Agency reports or records 5
According to Yan & Tourangeau (2008) the average time to complete one survey item is 10 seconds. Therefore one survey of 19 items will take 3-­‐4 minutes to complete. Measuring the impact of information technology systems on police practices 69 Agency records are measurements conducted by certain agencies or third parties (Fairfax County, 2007). Since certain agencies or third parties have already conducted these measurements, the information is readily available, accessible and reviewed by a third party. Agency records can simply entail the collection of ‘ hard data’ that has already been conducted by other agencies or third parties. But agency records could also entail comprehensive qualitative studies regarding surveys, questionnaires, interviews, observations conducted by these agencies or third parties. One of the main advantages of these suggested methods of data collection is that they provide no additional taxation on end-­‐user. Hence, unobtrusive records (as the name suggests) are unobtrusive while agency records are measurements that have already been conducted. The downside may be the effort of installing instruments and reviewing the data before extracting and plugging the data in the belevingsmonitor. However, in terms of taxation, the required training, completion time, and response rate these methods of data collection after being reviewed and extracted still are more effective than surveys (Fairfax County, 2007) (table 6.3-­‐3). This does not mean that these types of data collection methods should substitute the use of surveys as a whole. Surveys still remain an exceptional method in quantifying the subjective qualitative perceptions of end-­‐users in a resource effective way. What this research does propose is to complement the use of surveys with additional data derived from existing agency records or unobtrusive measures. The use of this sort of data is especially attractive considering that this data is readily available, enabling us to increase the amount of indicators while simultaneously decreasing taxation on front line workers by having them fill in shorter length surveys. This ultimately contributes to the criteria specified, which is more knowledge for informed decision making while minimizing taxation on end-­‐users and data collectors. 70 Measuring the impact of information technology systems on police practices Characteristic Unobtrusive records Taxation on end-­‐
user None Agency reports Surveys No additional taxation, None to some, depending on how the since measurements have survey is handled already been conducted Taxation on Low, depending on amount Low, needs review and Moderate to long depending on how the data-­‐collector of data needed and can be extraction by data survey is handled programmed once to collect collector from report to data automatically the belevingsmonitor. Table 6.3-­‐3: Effectiveness of data collection method in terms of taxation on end-­‐user and data collector (Fairfax County, 2007) RECOMMENDATIONS This research proposes the owner of the belevingsmonitor to critically reflect upon the data collection methods used for the belevingsmonitor. Whether information comes from recording instruments or surveys, ultimately it comes down to increasing informed decision-­‐making. Although the belevingsmonitor is implicitly oriented towards measurement of ‘beleving’, there is no reason why the belevingsmonitor should not contain additional data readily available that is not necessarily related to perceptions. The inclusion of data derived from alternative sources besides surveys become increasingly relevant considering the large amount of relevant indicators identified in this research and the trend of readily available digitalized data emerging from implementing OPP E&S and future IT implementations. Ultimately, if we want to increase knowledge of the above-­‐identified indicators while simultaneously reducing taxation on resources the use of data readily available becomes inevitable. Therefore, this research recommends the owner of the belevingsmonitor to include information and data readily available from current agency records or from future unobtrusive mechanical (or digital) instruments. Examples of data collection through agency reports or records: •
•
Productivity In the context of OPP E&S, the police force already conduct monthly or yearly reports on the productivity of each former district. These reports record the amount of execution tasks being completed per time period and also throughput times of execution tasks. With the advent of OPP E&S, the recording of amounts and throughput times of execution tasks, it is expected that the use of agency records will shift towards automated digital instruments. Administrative work In the context of OPP E&S, the police force has already conducted a qualitative study on the amount of time that police officers, coordinators and administrative workers have to spend on the administrative part on conducting execution tasks. This report can be reviewed and extracted for data providing insight on the proportion of time spend on administrative tasks and time spent and operative tasks. Examples of data collection through unobtrusive measures that can be programmed into newly implemented systems. Note that implementation of unobtrusive measures still necessitates the process of reviewing and extracting the credibility of the collected data in terms of completeness and accuracy. For example, an unobtrusive measure performing the correct measurement in a system that is not used by end-­‐users means that the data is not credible for use. •
Accuracy of information output Measuring the impact of information technology systems on police practices 71 Recording the number errors marked by supervisors when revising submitted reports (Groff & Mcewen, 2008) •
Accuracy of information input Number of reports rejected by supervisor for errors (Groff & Mcewen, 2008) •
Ease of use The number of steps for product use to complete what needs to be completed (Privitera, 2011) •
Time use Elapsed Time—taking of report through supervisor approval (Groff & Mcewen, 2008) Completeness of information output Number of data items describing incidents (Groff & Mcewen, 2008) •
•
Timeliness of information output Elapsed Time—taking of incident report to appearance in the database (Groff & Mcewen, 2008) •
Efficiency Proportion of tasks marked as successful per engaged visit or search effort (Groff & Mcewen, 2008) •
Communication Number of communications records with others (Groff & Mcewen, 2008). Number of call history queries (Groff & Mcewen, 2008). •
Readability Number of reports rejected by supervisor as illegible (Groff & Mcewen, 2008). •
Safety Patrol officer on-­‐duty injuries (Groff & Mcewen, 2008). 72 Measuring the impact of information technology systems on police practices 6.3.3 VALIDITY OF CONDUCTING BEFORE AND AFTER MEASUREMENT IN THE BELEVINGSMONITOR The belevingsmonitor was brought to life as a method to keep track of trends brought forward by individual IS subprojects. In order to keep track of this trend, the belevingsmonitor employs periodic issuance of online surveys to end-­‐users of the system before and after the implementation of an IS project. This periodic deployment of the belevingsmonitor is appropriate as long as the newly implemented system serves the same purpose and same end-­‐users. In the context of OPP E&S however, this research observed that the implementation of OPP E&S will have a significant shift in the type of end-­‐
users of the system. The type of end-­‐users is expected to shift from a majority of administrative and supervising workers with the old system to a majority of operative officers as end-­‐users of the new system. Administrative workers and supervisors can have different norms, values, reference points and levels of experience compared to operative officers. Especially in the context of OPP E&S, where currently operative officers themselves never have to use the systems. There is a risk that differences in values, ranks or scores measured before and after IS implementation may be caused by the differences between end-­‐users instead of the impact of the new system. OPP E&S is just the first subproject of OPP and many other subprojects are expected to experience the same shift in end-­‐users for the new system. With this trend to be persistent across all OPP projects, this issue will have implications for the reliability of conducting periodic measurement in the belevingsmonitor. Dependent on the resources available, the owner of the belevingsmonitor has the following options to consider pursuing before and after comparisons in the belevingsmonitor. •
•
•
Accept the inability to compare before and after measurements Accept the risk of differences in scores or ranks being caused by differences in norms, values and reference points of the samples taken. Reserve resources to assess the internal validity of comparing results from samples taken before and after implementation. This research proposes the following actions to review the reliability of comparing before and after measurements, specifically for OPP E&S and belevingsmonitor implementations for other future OPP projects. o Use supervisors as a control group In the context of OPP E&S, supervisors are a very small group of end-­‐users that have hands-­‐on experience and knowledge about the current systems, but are also expected to have hands-­‐on experience with the new system as well. This group of end-­‐users can be used as a control group. If the after implementation survey results of supervisors do not exhibit significant differences compared to survey results of operational officers, the before and after survey results can be considered as internally valid for comparison o Force administrative users to test the new system and use administrative workers as a control group An alternative would be to use administrative users as a control group to replace or supplement supervisors as a control group. By letting administrative users test the new system and filling in surveys, the results from administrative users can be used as a control. If the testing survey results of administrative workers do not exhibit significant differences compared to survey results of operational officers, the before and after survey results can be considered as internally valid for comparison. A disadvantage of Measuring the impact of information technology systems on police practices 73 this option however, is that organizing testing events with administrative workers is resource intensive, both for administrative workers and the data collector himself. Another disadvantage is that testing lacks prolonged use of the system. This results in the risk that end-­‐users will fill in surveys based on first impressions instead of practical experiences. In case the use of control groups fails in confirming internal validity, the owner of this research has the option to exclude before and after measurement or to increase reliance on non-­‐survey related methods as described in the previous section, such as unobtrusive digital measures or agency reports. 6.3.4 FUTURE SCENARIOS FOR MEASUREMENT This chapter proposed various factors for the owner of the belevingsmonitor to take into account regarding indicators to include in the belevingsmonitor, data collection methods and the ability to conduct before and after measurements. Based on these considerations, this chapter also provides recommendations for the owner to consider. Based on the factors to take into account and recommendations mentioned above, this research concludes this chapter with an illustration of four possible future scenarios of the belevingsmonitor and its implications of how decision-­‐making in these scenarios is affected. Scenario 1: Continuing the current state of belevingsmonitor In this scenario, the current construction of the belevingsmonitor will remain unchanged. Implications for effective decision-­‐making: •
•
•
Indicators: From the perspective of end-­‐users, the belevingsmonitor misses the opportunity to monitor very important indicators such as aspects of information quality (accuracy, completeness and timeliness) while devoting resources to the measurement of less relevant indicators such as social influence and facilitating conditions. Data collection: Maintaining the use of surveys for the belevingsmonitor means that no additional training will be needed for the data-­‐collector/analyst and that no adjustments are necessary for the current survey distribution process. However, the sole use of surveys misses the opportunity of gaining additional knowledge for informed decisions from data that is readily available. Before and after measurement: Continuing the current method of before and after measurements has the risk that sampled data to be invalid for conducting before and after comparisons. Scenario 2: An adjusted belevingsmonitor In this scenario, the belevingsmonitor will still utilize survey items issued before and after implementation of an IS. However, some additional adjustments will be made. •
•
Removal of survey items related to social influence and facilitating conditions Addition of the following survey items regarding information quality and time use o “The information I receive is often complete” Agree 00000 Disagree o “The information I receive is often accurate” Agree 00000 Disagree o “The information I receive is often on time” Agree 00000 Disagree o “What is the approximate time necessary to input or extract information from the system?” 74 Measuring the impact of information technology systems on police practices “What is the approximate proportion of time for execution tasks spend doing administrative tasks compared to operative tasks?” Establishing a control group to validate comparison of before and after measurements (supervisors or administrative users as control group) o
•
Implications for decision-­‐making: •
•
•
Indicators: The addition of survey items regarding information quality and time use means that decision-­‐making using the belevingsmonitor will be based off IT impact that matter the most (from a perspective of end-­‐users). At the same time, removal of less relevant indicators such as social influence and facilitating conditions means that the net taxation of end-­‐users will remain the same. Data collection: By maintaining the same method of data collection through surveys, minimal adjustments are necessary to the current process of the belevingsmonitor. The data collectors require no training and apart from a few new entries in the online survey distribution system, the distribution process remains the same. At the same time, limiting the belevingsmonitor to the sole use of surveys misses the opportunity of gaining additional knowledge from data that is readily available. Before and after measurement: By establishing a control group for survey results from a different sample of end-­‐users before implementation and after IS implementation, it can be validated whether comparison before and after IS implementation is possible. However, this adjustment only provides a method of validation and thus the risk of conducting invalid before and after comparisons of survey data still remains. Scenario 3: Adjusted belevingsmonitor + data already available from systems and reports In this scenario, the adjusted belevingsmonitor will be implemented along with inclusion of data available from systems and reports. As of now it is known that for OPP E&S, agency reports available regarding productivity of departments and proportion of time spent on execution tasks. Information from these reports can be reviewed en extracted into the belevingsmonitor. Considering that OPP E&S will digitize most information flows through system data, it is expected that loggings of steps needed to extract information, and records of information quality aspects to become be available when the new system is implemented. This system data should also be extracted into the belevingsmonitor. Examples of system data and agency report data are displayed in table 6.3-­‐3. Measuring the impact of information technology systems on police practices 75 Concept Productivity
Efficiency
Ease of use
Time use
Adminstrati
ve tasks
Accurate
input info
Accurate
output info
Complete
output info
Timely
output info
Unit of measurement Average amount of execution tasks completed per month/year Average turnaround time of a completed execution task cycle Man-­‐hours spent on execution-­‐
related tasks Measure Performance agency reports Amount of steps needed from extraction of information to a completed overview of an individual execution task Time spent behind IS to input or output information. Record Average ratio time spent on administrative work / operative work for execution tasks Reported faults Performance reports Reported faults Random sampling/ Record Average amount of fields displayed Random sampling/ Record Average age of information (compared to date of access) Random sampling/ Record Performance agency reports Record Random sampling/ Record Table 6.3-­‐4 Data collection example for OPP E&S measurement through system and report data Implications for decision-­‐making: •
•
•
Indicators: The addition of data already available automatically entails that the belevingsmonitor will contain more indicators providing richer information for decision-­‐
making. At the same time, the inclusion of these indicators based on available data means that the taxation on end-­‐users will remain the same. Data collection: Including data already available from systems and reports requires significant effort on the data-­‐collectors part to identify these sources and to design a process to include this data into the belevingsmonitor. The effort for this reviewing and extracting process depends whether these sources are reports or instruments. In case of reports, the data collector has to perform review and extraction every time such a record is published. For instruments however, the data collector can program a script for automatic extraction into the system and review the workings of this script once. Before and after measurement: Data from systems or reports are not exposed to subjectivity and characteristics of end-­‐users filling in surveys. Therefore, data from systems or reports can be used as a complementary source of insight, or in case survey comparisons are found to be invalid, be used as a substitute source of insight for before and after comparisons. However, a decision-­‐maker should always critically reflect upon the cause of trends in system and report data, since a trend can never be isolated to causation of IT impact alone (due to the emergent nature of IT impact). Scenario 4: Balanced scorecards: an IT-­‐impact monitor 76 Measuring the impact of information technology systems on police practices In this scenario, a radical change to the belevingsmonitor is implemented. Instead of using UTAUT or any other technology acceptance model as the foundation of the monitor, a balanced scorecard foundation is build based on the findings identified in this research. Balanced scorecards is a performance measurement principle originated by Drs. Robert Kaplan and David Norton as a performance measurement framework that added strategic non-­‐financial measurement goals to traditional financial measurement goals to give decision makers a more ‘balanced’ view of the performance of their organization. This research will not recommend to literally adopting the balanced scorecard framework by Kaplan. Rather, this research recommends the owner of the belevingsmonitor to apply basic principles underlying the balanced scorecard framework. These basic principles include the definition of important perspectives for measurement, the selection of indicators for each perspectives and the construction of data collection methods. Based on the findings in this research, five generic measurement perspectives on IT impact on police practices can be identified. These five measurement perspectives are preliminary aspects regarding IT use, the input of information, the output of information, application of information and outcome related to using information technology systems. For each measurement perspective we included the most important indicators for measurement. How these perspectives including indicators are identified is detailed in appendix H. Preliminary
Input
Output
Complete
informatio
n
Ease of
use
Time use
Accurate
informatio
n
Accurate
informatio
n
Timely
informatio
n
Acceptan
ce
Application
Communi
cation
Task
innovatio
n
Adminstr
ative
tasks
Preserve
knowledg
e
Outcome
Proactivit
y
Productiv
ity
Managem
ent
control
Efficiency
Figure 6.3-­‐1 Balanced scorecard framework including indicators to represent measurement of IT impact on police practices As this research assumes contingency, the presented balanced scorecard is not the ultimate best way to measure IT impact. Since this research identified 42 relevant indicators of which for all 42 indicators both survey and mechanical data can be collected for, in the end, it is the decision of the owner to balance more information against increased taxation on resources. The framework presented in figure 6.3-­‐1 can serve as a benchmark for the owner to make adjustments from. The owner can choose to include more indicators or exclude indicators dependent on the availability of resources, time and desired taxation on end-­‐users. This research recommends including at least one indicator for each of the five generic measurement perspectives. Concept Productivity
Efficiency
Unit of measurement Average amount of execution tasks completed per month/year Average turnaround time of a completed execution task cycle Man-­‐hours spent on execution-­‐
related tasks Measure Performance agency reports Survey item:” How much time do you Survey approximation Performance agency reports Measuring the impact of information technology systems on police practices 77 Ease of use
Time use
Adminstrati
ve tasks
Proactivity
Task
innovation
Management
control
Communicati
on
Acceptance
Preserve
knowledge
Accurate
input info
Accurate
output info
Complete
output info
Timely
output info
spent behind your pc for execution related tasks?” Amount of steps needed from extraction of information to a completed overview of an individual execution task Time spent behind IS to input or output information. Record Observation Average ratio time spent on administrative work / operative work for execution tasks Survey item: “The extent that IT assists me in performing tasks more proactively” Survey item: “The extent that IT assists me in trying out new ideas in performing tasks” Survey item “The extent that IT assists me in managing and controlling myself and my team” Survey item “The extent that IT assists me in communicating in my team” Survey item: “The extent that IT is accepted by me or my team” Performance reports Survey approximation Survey item: “The extent that IT preserves or assists in the preservation or improvement of explicit and implicit policing knowledge and expertise” Reported faults Survey ratings Reported faults Random sampling/ Record Average amount of fields displayed Random sampling/ Record Average age of information (compared to date of access) Random sampling/ Record Survey ratings Survey ratings Survey ratings Survey ratings Survey ratings Random sampling/ Record Table 6.3-­‐5 Sample operationalisation of the 15 indicators of the balanced scorecard Implications for decision-­‐making: •
•
Indicators: Since the indicators in this scenario are directly selected based on findings of this research it can be stated that this scenario would contain the most ‘ideal’ mixture of indicators from a perspective of end-­‐users. Data collection: The principle of using a balanced scorecard instead of a locked measurement model implies that data collection can be implemented in a flexible manner. The use of generic perspectives and instead of statically defined measurement models allows for the design where customized perspectives that do not necessarily have to relate, can be added or removed in accordance to the measurement goal of the owner of the belevingsmonitor. Since the use of a balanced scorecard is also not limited to ‘perceptions’ one can mix and match methods of data collections in accordance to the owners specifications and or availability. In line with availability of resources, the owner can also choose to implement more or less indicators for each of the perspectives included in the scorecard. 78 Measuring the impact of information technology systems on police practices •
Before and after measurement: By having a mixture of objective and subjective data, this measurement scenario does not have to rely on subjective data alone to make before and after comparisons. 6.4 CONCLUSION With a focus on practical use, this chapter presented final considerations and recommendations towards the design of a measurement tool. Since the police force already has a measurement tool defined as the belevingsmonitor, the considerations and recommendations proposed in this chapter were reflected upon the current state of the belevingsmonitor. Based on this comparison, this chapter proposed three elements of the belevingsmonitor to consider, which are the indicators to include, the data collection and the viability of comparing before and after measurements. For each of the considerations discussed, this research also included recommendations to improve the belevingsmonitor. Since the belevingsmonitor is planned for measurement in the context of OPP E&S, the considerations and recommendations were oriented towards use for OPP E&S in mind. Based off these considerations and recommendations, this chapter included four example scenarios of future measurement and what the implications towards effective decision making is when conducting such measurement scenarios. An overview of the four possible future measurement scenarios is briefly discussed in the table below. 1. Current state belevingsmontor 2. Adjusted belevingsmonitor Adjust
ments None Maintain data collection through surveys. Replace survey items relating to facilitating conditions and social influence by survey items relating to information quality and time use 3. Adjusted belevingsmonitor + existing data from systems and reports Include adjustments from scenario 2. Include data and information available and obtainable from reports and systems. 4. Balanced scorecards Replace belevingsmonitor by a balanced scorecard consisting of five perspectives with at least one indicator for each perspective. Measurement through a combination of survey items, report data and system data Include check on control group to validate ability to conduct before and after measurements. Implic
ations for decisi
on-­‐
makin
g + Minimum adjustments necessary -­‐ Adjustments needed to survey design -­‐ Adjustments needed to survey design -­‐ Measuring less relevant indicators while missing more important indicators such as information quality and time use -­‐ Risk of invalid comparison before and after survey data -­‐ Adjustments necessary to implement review and extract processes for system data and report data -­‐ Sole use of surveys misses out on opportunity of using readily available data for +Having a control group provides a control for the validity of before and after comparisons of subjective data + Creating a more relevant +Provides additional knowledge for decision-­‐
maker without extra taxation on end-­‐user. -­‐ Significant changes necessary to the complete structure of the belevingsmonitor. + A balanced scorecard structure is more flexible for adjustments than using a scientific measurement model +Includes the ‘ideal’ mixture of indicators that are most important from the Measuring the impact of information technology systems on police practices 79 decision making -­‐ Risk of invalid comparison before and after survey data survey for decision making by omitting less relevant indicators and replacing it by more relevant indicators +All benefits from scenario 2 + Survey length remains approximately the same, thus taxation on end-­‐user remains the same as well. 80 Measuring the impact of information technology systems on police practices perspective of end-­‐users + Mixture of data from subjective and objective sources decreases the risk of not being able to make before and after comparisons. 7. CONCLUSION The final chapter of this study starts by answering the research (sub-­‐) questions. Afterwards, the limitations of this research were analyzed. In extension, further research in this field is recommended. The final part of this chapter contains the reflections and learning points on this research project. 7.1 ANSWERING THE RESEARCH QUESTIONS This section will answer all of the research questions formulated in the first chapter of this research. What is the domain of the research objective and the corresponding problems within this domain? The domain of this research is the impact of technological change on police practices, more specifically information technology. Information technology refers to all the technology used to process information including information systems. The level of impact analysis for this research is police practices at the level of individual end-­‐users of the information technology systems. The corresponding problem relates to combining the domain and the objective. As discussed in chapter two, to develop a measurement method, concepts that should be measured have to be identified. Chapter two discussed the following four problems with defining a measurable concept for police practices. •
•
•
•
Police practices are difficult to define as a singular concept The problem with defining the concept of police practices as a whole is that it cannot be captured by a singular concept; it is the accumulation of many practices that interrelate with each other. Police practices cannot be just substituted by a simpler concept Because police practices is the accumulation of many concepts that interrelate with each other, simplifying the concept of police practices into simpler substitute concepts such as crime rate or productivity would also not do the concept of police practices justice. These concepts only capture some aspects of what police practices entail. Different stakeholders perceive the concept of police practices differently Even if it is possible to identify all concepts relating to police practices from one point of view, individuals with different backgrounds and philosophical positions will subscribe totally different concepts to represent police practices in their view. Due to the emergent property of technological change, it is impossible to isolate concepts of police practices impacted by technology alone. Technology operates in a social context, and technological change emerges from the interaction between technology and people. So in the context of observing or measuring technological change, it is not possible to isolate the technological or the human contribution part out of a measurable concept. What is systems thinking and how can it be applied on the development of a measurement tool? Due to the increasing fragmentation of science into sub-­‐groups, a new research approach called systems thinking was developed by multiple researchers in the 1950s concerning the integration of these disciplines. Systems thinking entails an approach with a focus on understanding how things work, where effects or outputs of any system are dependent on the interaction of its parts where Measuring the impact of information technology systems on police practices 81 studying the parts in isolation will not provide an accurate picture of the system. Considering the problems related to the domain, this research applied the system thinking approach by: •
•
•
Not trying to force singular or few concepts to represent police practices Consider multi-­‐perspective property of concepts representing police practices Acknowledge emergent property of technological change, and thereby not trying to force identification of predictive causal relationships. This research applies the system thinking approach primarily for identifying the landscape of the police practices landscape impacted by technology. This identification process consists of documenting all relevant concepts as complete as possible and its possible relationships in a mental map and afterwards deriving the most relevant indicators from this mental map. By providing several operationalization scenarios of these indicators, a measurement tool can be developed. What are the concepts that should be measured? The findings of this research resulted in a mental map consisting of 42 relevant concepts that represent police practices impacted by information technology. All these 42 concepts can be considered relevant for measurement. Considering the resource intensiveness of measuring such a large amount of concepts, these 42 concepts have been filtered down to 15 most relevant indicators These 15 indicators are concepts of which end-­‐user consider them to be the most important indicators to represent policework impacted by information technology. How can these concepts be operationalized? No single operationalization method is indefinitely better than another type of operationalization method. Therefore, it is not possible to come up with a single or best way to operationalize measurement of relevant concepts. This research proposes four possible ways of operationalization scenarios in respect to the current design of the measurement tool of the police force, the belevingsmonitor. The scenarios provided range from small adjustments to complete revamps of the measurement method to better reflect the findings of this research. Ultimately, these four measurement scenarios serve as assistance for the owner of the belevingsmonitor to decide upon how extensive he is willing to make changes to the current design of the belevingsmonitor. 7.2 DISCUSSION OF FINDINGS The discussion section takes a broad view of the results of this research and puts it in a wider context. A generalized review and expectations of the findings within this research will be discussed. Findings in this research will also be compared to findings found in other studies. WHAT ARE THE GOAL, GENERAL FINDINGS AND EXPECTATIONS OF THIS RESEARCH? The purpose of the study was to develop a measurement method that is able to map the individual impact of information technology on police practices. The findings of this research resulted in the identification of 42 concepts, which were filtered to 15 most relevant indicators and operationalized for inclusion in an ‘ideal’ final measurement tool. Considering that the police force already has a measurement tool called the belevingsmonitor, this research also proposed few other alternative measurement scenarios next to the final ‘ideal’ measurement tool we presented. Although this research focuses on completeness of how the technology and police practice landscape looks like, a general finding in this research is that the concept of receiving accurate, complete and timely has been found to be a very prevalent and obvious reoccurring topic in this research. As 82 Measuring the impact of information technology systems on police practices shown in the mental map, the heat map and the ranked list of concepts, these three concepts were ranked the most relevant. Another interesting point is that findings of this research shows that the landscape of policing is more or less similar to the landscape of a business context. The expectations for this research were that findings would lead to the identification of concepts that are in stark contrast with a business context, such as safety, consistency and situational awareness. Although topics regarding safety, awareness and consistency did form a topic of discussion and narratives during interviews, when it came down to ranking the various concepts on utmost relevancy, the typical business related concepts still made it to the top of the list. HOW DO FINDINGS FROM THIS RESEARCH DIFFER FINDINGS FROM OTHER RESEARCH DEDICATED TO MAPPING INDIVIDUAL IMPACT? Findings in this research seem to contradict findings by Gable, Sedera, & Chan (2003) and Torkzadeh & Doll (1999), two authors that also focused research on identifying concepts for individual impact of information technology. This research shows that the concepts learning, awareness/recall, decision effectiveness proposed by Gable, Sedera, & Chan (2003) and customer satisfaction proposed by Torkzadeh & Doll (1999) not to be the most important concepts. Although these concepts have been confirmed to some extent during the interviews, the relative importance of these concepts are far less compared to other concepts that were identified. Furthermore, the findings of this research also suggest far more concepts compared to the amount of concepts proposed by Gable et al. and Torkzadeh & Doll. A possible explanation for this difference is possibly due to the policing context of this research, which is different from the context used by Gable et al. and Torkzadeh & Doll. Another possible explanation is that the concepts proposed by Gable et al. and Torkzadeh & Doll were identifed based on expert judgement while the proposed measures in this research are identified based on end-­‐user opinion and experience. This leaveas us with the question whether the ‘expert’ or the end-­‐user should be considered right. HOW DOES THIS RESEARCH RELATE TO TAM AND THE IS-­‐SUCCESS MODEL? TAM, IS-­‐success and the approach in this research are fundamentally three different approaches in understanding the interaction between society and technology. This fundamental difference lies in the fact that TAM is primarily a model primarily to predict acceptance and use. IS-­‐success was built with the intention to predict successful implementation of IS. (Predicting success was the trigger for Delone to develop this model, but the model has also evolved to include elements of process approach as well). TAM and IS focus on predicting certain phenomena and can therefore be classified as variance approaches. The model in this research however was not meant to predict a certain phenomena, but rather to understand a phenomenon by trying to expose the underlying concepts and structure of a system that we defined as the police practices system. Although fundamentally different in purposes and goals, findings from this research does not seem to contradict technology acceptance and IS-­‐success, but rather provides supplementary findings to these two models. •
As noted by authors of technology acceptance models such as TAM or UTAUT: acceptance of information technology is necessary for users to be willing to use information technology. Findings in this research do not necessarily contradict this notion. What this research does show however is that acceptance is just one little piece of the puzzle and not the key to unlocking the door. It becomes problematic if researchers or practitioners solely spend their efforts on understanding or predicting the concept of acceptance alone. Since this research Measuring the impact of information technology systems on police practices 83 •
shows, next to acceptance, there are 41 other concepts that are relevant in their contribution to technological change and positive impact on police practices. The IS-­‐success model has the goal to define, understand and predict system success. The authors defined IS-­‐success to be dependent on six concepts, which are system quality, service quality, user satisfaction, use, individual impact and organizational impact. Speaking in IS-­‐terms, this research can be seen as measuring individual impact. Therefore, this research supplements the IS-­‐success model by providing relevant concepts that can be used to measure individual impact. 7.3 LIMITATIONS AND RECCOMENDATIONS FOR FURTHER RESEARCH One of the limitations of this research is related to the data collection method. As discussed in the design chapter, the interviews where held around a central topic in order to provide a consistent reference point for interviewees provide narratives. This central topic was that of a newly to-­‐be introduced information system called OPP E&S. Since OPP E&S is still under heavy development, the exact changes that will be introduced with OPP E&S are still uncertain. How exactly OPP E&S will impact practices of interviewees remained a topic of speculation. While end-­‐users provided most narratives based on current experience, there was some level of speculation involved. Because of this speculation, there is a possibility that topics that are not relevant at this moment will become more apparent after the implementation of OPP E&S. The same limitation applies to the card sorting method. Although the card sorting sessions were issued to a wider set of users other than OPP E&S users, these users are not yet familiar with future IS implementations and will rank concepts based on what is relevant for the current situation. Because of this, there is a possibility that topics that are ranked low in the current situation could become more relevant in a near future. Therefore, the possibility of changing relevancy of topics should be taken in consideration in future research. The landscape of information technology changes rapidly and therefore it is recommended to verify or extend the amount of identified concepts from this research after future IS implementations. Possibilities in doing this can range from conducting interviews or card sorting sessions again after IS implementations Another limitation is related to the identification of most relevant indicators. Common system thinking approach dictates the identification of indicators through simulation. Due to time and resource constraints however, this research was not able to perform the extensive research to properly identify validated relationships between the identified concepts. The relationships depicted in the mental map was only indicative and not credible enough for simulations (incomplete and no feedback loops). Furthermore, simulation requires starting values of which the data is not available yet. Therefore instead, this research relied on the ratings provided by end-­‐users to pick most relevant indicators. Relying on these end-­‐user ratings however, can sometimes be problematic since according to system thinking since “ sometimes people’s intuitivism of where most important elements are, are sometimes wrong”. Therefore, further research is recommended in continuing the development of the mental map in this research towards a simulation model and identifying most important concepts through the simulation instead. The findings can then be compared to the indicators in this research to provide insight in the ability of end-­‐users to identify most relevant indicator points. 84 Measuring the impact of information technology systems on police practices Furthermore, the operationalization scenarios proposed for the measurement tool could not be tested for appropriateness due to time and resource constraints. Therefore, it would be interesting if further research can be dedicated into testing and comparing how different operationalization scenarios would lead to different results and base proposition for operationalization based on these differences. The last limitation that will be discussed is the topic of subjective bias. Although a lot of effort has been spent to provide internal validity and transparency, in the end this research is explorative and qualitative in nature. Therefore, it relies very little on existing scientific rigor and more on qualitative empirical data and observatory insights. Due to this qualitative and explorative nature, this research should be considered as a starting point in an iterative research trajectory of constant tweaking, adjusting and improvements in developing a measurement tool. 7.4 REFLECTION Reflecting on this research, the main difficulty was finding the right theoretical path to walk on towards the final goal. I began this research by using technology acceptance models, IS-­‐success models and even technology task-­‐fit models as a starting point. My initial goal was to contribute to one of these models by adding additional predictive factors to these models and this notion of certain factors predicting other factors seemed to stick. Further down the road and having spoken to different kind of people within the organization increased my understanding of the policing domain and how it is affected by technology. More and more did it seem to align with the systems thinking approach that we have learned during the bachelor and master courses. Having realized and having embraced the system thinking way of looking at the subject I started to let go of the focus on prediction and causal relationships. Instead the research was focused on exploring the semantics of the police practices system and mapping the trees within this forest. The difficult part however, is explaining how the system thinking approach is different from other approaches and finding the right designations for what I was doing. Lucky enough, I was able to find that one article (Burton-­‐Jones et al., 2011) that explained differences between regular and system thinking approaches in an IS-­‐use context. Another topic was my underestimation on using qualitative studies especially in an exploratory context. Having to make qualitative judgments for the identification of concepts was a gruesome process. It is very difficult to keep track of what various authors and interviewees have to say about certain topics. Also having to align the various statements and narratives based on your own qualitative judgment tends to create uncertainty that is less comforting then the concrete numbers and values that we are used too during our studies. Luckily, the quantitative sorting sessions provided a lot of insights beyond validation of the mental map and identification of indicators. The sorting sessions provided tons of insights that I could relate back to the narratives that I have heard during the interviews. When the card sorting data was completed and analyzed, it felt as if all puzzle pieces fell together. This experience has really thought me the power of using mixed methods, as qualitative methods and qualitative methods assist each other. Measuring the impact of information technology systems on police practices 85 REFERENCES Abugabah, A. J., Sanzogni, L., & Poropat, A. E. (2009). The impact of information systems on user performance: A critical review and theoretical model. In International Conference on Computer Science and Engineering. WASET. Algemene rekenkamer. (2011). ICT politie 2010. The Hague. Alter, S. (1999). A General, Yet Useful Theory of Information Systems. Communications of the Association for Information Systems, 1(13). Bauer, M. (1996). The narrative interview: Comments on a technique for qualitative data collection. London School of Economics and Political Science. London: London School of Economics and Political Science. Bertalanffy, L. Von. (1950). An outline of General System Theory. The British Journal for the Philosophy of Science, I(2), 134–165. Boehm, B. (1988). A spiral model of software development and enhancement. Computer, 21(5), 61 – 72. Boulding, K. E. (1956). General Systems Theory—The Skeleton of Science. Management Science, 2(3), 197–208. Briggs, R. (2001). The technology transition model: A key to self-­‐sustaining and growing communities of GSS users. System Sciences, 2009. HICSS ’09. 42nd Hawaii International Conference. Briggs, R. (2003). Collaboration engineering with ThinkLets to pursue sustained success with group support systems. Journal of Management Information Systems, 19(4), 31–64. Brown, M. (2014). Revisiting the IT Productivity Paradox. The American Review of Public Administration. Brown, S., & Peterson, R. (1994). The effect of effort on sales performance and job satisfaction. The Journal of Marketing, 58(2), 70–80. Brynjolfsson, E., & Yang, S. (1996). Information Technology and Productivity : A Review of the Literature. Advances in Computers, 43, 179–214. Burton-­‐Jones, A., Mclean, E. R., & Monod, E. (2011). On approaches to building theories: Process, variance and systems. European Journal of Information Systems. Caballero, I., Verbo, E., Calero, C., & Piattini, M. (2009). A Data Quality Measurement Information Model Based On ISO/IEC 15939. Latin America Transactions IEEE, 7(3), 361 – 368. Chan, J., Brereton, D., Legosz, M., & Doran, S. (2001). E-­‐policing: The impact of information technology on police practices. Queensland criminal justic commission. 86 Measuring the impact of information technology systems on police practices Chuttur, M. (2009). Overview of the Technology Acceptance Model: Origins, Developments and Future Directions. Working Papers on Information Systems, 9(37). Colvin, C. A., & Goh, A. (2005). Validation of the technology acceptance model for police. Journal of Criminal Justice, 33(1), 89–95. Creswell, J. W. (2012). Qualitative inquiry and research design: Choosing among five approaches. Sage publications. Cyrus, A. W. (1991). Measuring the effectiveness of information systems. Mississippi State University. Daskalakis, S., & Mantas, J. (2008). Evaluating the impact of a service-­‐oriented framework for healthcare interoperability. Studies in Health Technology and Informatics, 136, 285–90. Davis, F. D. (1986). A technology acceptance model for empirically testing new end-­‐user information systems: Theory and results. Massachusetts Institute of Technology. Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(3), 319–340. DeLone, W. H., & McLean, E. R. (1992). Information Systems Success: The Quest for the Dependent Variable. Information Systems Research, 3(1), 60–95. doi:10.1287/isre.3.1.60 Delone, W. H., & McLean, E. R. (2003). The DeLone and McLean model of information systems success: a ten-­‐year update. Journal of Management Information Systems, 19(4), 9–30. Devaraj, S., & Kohli, R. (2003). Performance impacts of information technology: is actual usage the missing link? Management Science, 49(3), 273–289. Dishaw, M. T., & Strong, D. M. (1999). Extending the technology acceptance model with task–
technology fit constructs. Information & Management, 36(1), 9–21. doi:10.1016/S0378-­‐
7206(98)00101-­‐3 Elasy, T. A., & Gaddy, G. (1998). Measuring subjective outcomes: rethinking reliability and validity. Journal of General Internal Medicine, 13(11), 757–61. Fairfax County. (2007). Manual for Data Collection for Performance Measurement (p. 27). Forrester, J. W. (1973). Confidence in models of social behavior with emphasis on system dynamics models. System Dynamics Group Working Paper. Fryk, P. (2007). Effects of Digitized Information Flows in Health Care. Linköping University. Gable, G., Sedera, D., & Chan, T. (2003). Enterprise Systems Success: A Measurement Model. In Twenty-­‐Fourth International Conference on Information Systems. Seattle, USA. Gable, G., Sedera, D., & Chan, T. (2008). Re-­‐conceptualizing Information System Success : the IS-­‐
Impact Measurement Model. Journal of the Association for Information Systems, 9(7), 377–408. Galve-­‐Górriz, A. (2007). Information Technology, Complementarities and Three Measures of Organizational Performance: Empirical Evidence from Spain. Journal of Information Technology Impact, 7(1), 43–58. Measuring the impact of information technology systems on police practices 87 Garicano, L., & Heaton, P. (2006). Computing Crime : Information Technology , Police Effectiveness , and the Organization of Policing. CEPR Discussion Paper No. 5837. Goodhue, D. L., & Thompson, R. L. (1995). Task-­‐Technology Fit and Individual Performance. MIS Quarterly, 19(2), 213. doi:10.2307/249689 Groff, E., & Mcewen, T. (2008). Identifying and Measuring the Effects of Information Technologies on Law Enforcement Agencies. Hekim, H., Gul, S., & Akcam, B. (2013). Police Use of information technologies in Criminal Investigations. European Scientific Journal, 9(4), 221–240. Hoefnagel, R. (2013). Onderzoeksvoorstel : De IV Belevingsmonitor : Zicht op IV beleving politiemensen. Hoy, W. K. (2010). Quantitative Research in Education: A Primer (p. 160). SAGE Publications. doi:http://dx.doi.org/10.4135/9781452272061 Hu, P. J.-­‐H., Chen, H., Hu, H., Larson, C., & Butierez, C. (2011). Law enforcement officers’ acceptance of advanced e-­‐government technology: A survey study of COPLINK Mobile. Electronic Commerce Research and Applications, 10(1), 6–16. doi:10.1016/j.elerap.2010.06.002 IDC. (2013). IDC Predictions 2013 : Competing on the 3rd Platform. Framingham, USA. IEEE. (1998). IEEE Guide for Information Technology -­‐ System Definition -­‐ Concept of Operations (ConOps) Document. IEEE Computer Society, 1–24. doi:10.1109/IEEESTD.1998.89424 Karr-­‐Wisniewski, P., & Lu, Y. (2010). When more is too much: Operationalizing technology overload and exploring its impact on knowledge worker productivity. Computers in Human Behavior, 26(5), 1061–1072. doi:10.1016/j.chb.2010.03.008 Kerlinger, F. N., & Lee, H. B. (1968). Foundations of behavioral research: Educational and psychological inquiry. The Indian Journal of Political Science, 29(4), 409–411. Kinzig, A. P., Ehrlich, P. R., Alston, L. J., Arrow, K., Barrett, S., Buchman, T. G., … Saari, D. (2013). Social Norms and Global Environmental Challenges: The Complex Interaction of Behaviors, Values, and Policy. Bioscience, 63(3), 164–175. doi:10.1525/bio.2013.63.3.5 Koch, T. (1994). Beyond measurement: fourth-­‐generation evaluation in nursing. Journal of Advanced Nursing, 20(6), 1148–55. Lampel, J. (2004). A Note on Research Design. Laszlo, A., & Krippner, S. (1998). Systems theories: Their origins, foundations, and development. Elsevier Sience, 3, 47–76. Lee, H., Kim, T., & Choi, J. (2012). A Study on the Factors Affecting Smart Phone Application Acceptance. In 3rd International Conference on e-­‐Education (Vol. 27). Lee, H., & Whitley, E. A. (2002). Time and Information Technology: Temporal Impacts on Individuals, Organizations, and Society. The Information Society, 18(4), 235–240. doi:10.1080/01972240290075084 88 Measuring the impact of information technology systems on police practices Lind, R. C., & Lipsky, J. P. (1971). The Measurement of Police Output: Conceptual Issues and Alternative Approaches. Law and Contemporary Problems, 36(4), 566–588. doi:10.2307/1190935 Lucas, H. C., & Nielsen, N. R. (1980). The Impact of the Mode of Information Presentation on Learning and Performance. Management Science, 26(10), 982 – 993. Lucas, H., & Spitler, V. (1999). Technology use and performance: a field study of broker workstations. Decision Sciences. Maani, K., & Cavana, R. . (2007). Systems Thinking, Systems Dynamcs: Managing change and complexity. Auckland: Prentice Hall. Maier, C., Laumer, S., Eckhardt, A., & Weitzel, T. (2013). Analyzing the impact of HRIS implementations on HR personnel’s job satisfaction and turnover intention. The Journal of Strategic Information Systems, 22(3), 193–207. doi:10.1016/j.jsis.2012.09.001 Maltz, M. D., Gordon, A. C., & Friedman, W. (1991). Mapping crime in its community setting: Event geography analysis (p. 194). Nabu Press. Manning, P. K. (1992). Information Technologies and the Police. Crime and Justice, 15(1992), 349–398. Markus, M. L., & Robey, D. (1988). Information technology and organizational change: causal structure in theory and research. Management Science, 34(5), 583–598. Mason, R. O. (1978). Measuring information output: A communication systems approach. Information & Management, 1(4), 219–234. doi:10.1016/0378-­‐7206(78)90028-­‐9 Meadows, D. (1999). Leverage Points: Places to Intervene in a System. The Sustainability Institute. The Sustainability Institute. Merchant, K., Stringer, C., & Theivananthampillai, P. (2009). Relationships between objective and subjective performance ratings (No. 17). Messerschmitt, D. G. (1996). The convergence of telecommunications and computing: What are the implications today? Proceedings of the IEEE, 84(8), 1167 – 1186. Ministerie V&J. (2011). Aanvalsprogramma Informatievoorziening Politie 2011-­‐2014. Nunn, S., & Quinet, K. (2002). Evaluating the effects of information technology on problem-­‐oriented-­‐
policing. If it doesn’t fit, must we quit? Evaluation Review, 26(1), 81–108. OECD. (2008). Measuring the impacts of ICT using official statistics. Olugbode, M., Elbeltagi, I., Simmons, M., & Biss, T. (2008). The Effect of Information Systems on Firm Performance and Profitability Using a Case-­‐Study Approach. The Electronic Journal Information Systems Evaluation, 11(1), 11–16. Orlikowski, W. J. (1996). Improvising organizational transformation over time: A situated change perspective. Information Systems Research, 7(1), 63–92. Orlikowski, W. J., & Gash, D. C. (1994). Technological frames: making sense of information technology in organizations. ACM Transactions on Information Systems, 12(2), 174–207. doi:10.1145/196734.196745 Measuring the impact of information technology systems on police practices 89 Paré, G., Bourdeau, S., Marsan, J., Nach, H., & Shuraida, S. (2008). Re-­‐examining the causal structure of information technology impact research. European Journal of Information Systems, 17(4), 403–416. Pelgrum, W., & Law, N. (2003). ICT in education around the world: Trends, problems and prospects. Petter, S., DeLone, W., & McLean, E. (2008). Measuring information systems success: models, dimensions, measures, and interrelationships. European Journal of Information Systems, 17(3), 236–263. doi:10.1057/ejis.2008.15 Pica, D., & Sørensen, C. (2004). On Mobile Technology in Context : Exploring Police Work. Journal of Computing and Information Technology, 12(4), 287–295. Pritchard, R. D. (1990). Measuring and improving organizational productivity: A practical guide. Greenwood Publishing Group. Privitera, M. B. (2011). What is ease of use? IDSA. Retrieved from http://www.idsa.org/what-­‐ease-­‐use Robey, D., & Sahay, S. (1996). Transforming work through information technology: A comparative case study of geographic information systems in county government. Information Systems Research, 7(1), 93–110. Rosenberg, N. (1994). Exploring the black box: Technology, economics, and history. Cambridge University Press. Rothstein, J. (1989). On Defining Subjective and Objective Measurements. Physical Therapy, 69(7), 577–579. Saviak, J. (2007). An investigation into the predictors of adoption and utilization of information-­‐
sharing networks by local law enforcement in three states. Schaik, F. D. van. (1990). Effectiveness of decision support systems. Technische Universiteit Delft. Sekaran, U. (2006). Research methods for business: A skill building approach (Vol. 51, pp. 51–6761–
51–6761). John Wiley & Sons. doi:10.5860/CHOICE.51-­‐6761 Senge, P. M., Cambron-­‐McCabe, N., Lucas, T., Smith, B., & Dutton, J. (2012). Schools that learn (updated and revised): A fifth discipline fieldbook for educators, parents, and everyone who cares about education. Random House LLC. Shapira, P., & Rosenfeld, S. (1996). An overview of technology diffusion policies and programs to enhance the technological absorptive capabilities of small and medium enterprises. In Background paper prepared for the Organization for Economic Cooperation and Development Directorate for Science, Technology and Industry. Singh, R., & Kroesen, O. (n.d.). Shaping Future Green Cities : LEDs Technology adoption as an option for India, 1–16. Solow, R. M. (1987). Second thoughts on growth theory. In Employment and Growth: Issues for the 1980s (pp. 13–28). Springer. Staron, M., & Meding, W. (2009). Using models to develop measurement systems: a method and its industrial use. In A. Abran, R. Braungarten, R. R. Dumke, J. J. Cuadrado-­‐Gallego, & J. Brunekreef (Eds.), Software Process and Product Measurement (pp. 212–226). Springer Berlin Heidelberg. 90 Measuring the impact of information technology systems on police practices Surry, D. W. (2008). Impact of Technology. University of South Alabama. Tatar, E. L. (2011). The importance of measuring individual performance to increase organizational performance. Journal of Defenses Resources Management, 1(2), 113–118. Torkzadeh, G., & Doll, W. . (1999). The development of a tool for measuring the perceived impact of information technology on work. The International Journal of Management Science, 27(3), 327–
339. doi:10.1016/S0305-­‐0483(98)00049-­‐8 UNCTAD. (2011). Measuring the Impacts of Information and Communication Technology for Development Measuring the Impacts of Information and Communication Technology for Development. United Nations. (2005). Guide to Producing Statistics on Time Use: Measuring paid and unpaid work. Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D. (2003). User Acceptance of Information Technology: Toward a Unified View. MIS Quarterly, 27(3), 425–478. Wang, R., & Strong, D. (1996). Beyond accuracy: What data quality means to data consumers. Journal of Management Information Systems, 12(4), 5–33. Watson, C. J., & Driver, R. W. (1983). The Influence of Computer Graphics on the Recall of Information. MIS Quarterly, 7(1), 45–53. doi:10.2307/249076 Wessels, M., & Timmermans, J. (2013). Project Initiatie Document ( PID ) Operationeel Politie Platform : Executie & Signalering. Williams, R., & Edge, D. (1996). The social shaping of technology. Research Policy, 25, 856–899. Yalcinkaya, R. (2007). Police officers’ adoption of information technology: A case study of the Turkish POLNET system. University of North Texas. Yan, T., & Tourangeau, R. (2008). Fast times and easy questions: the effects of age, experience and question complexity on web survey response times. Applied Cognitive Psychology, 22(1), 51–68. doi:10.1002/acp.1331 Zhou, T., Lu, Y., & Wang, B. (2010). Integrating TTF and UTAUT to explain mobile banking user adoption. Computers in Human Behavior, 26(4), 760–767. doi:10.1016/j.chb.2010.01.013 Measuring the impact of information technology systems on police practices 91 APPENDIX A: INTERVIEW STATEMENTS This appendix contains statements derived from the interviews categorized per identified concept. This detailed table serves as a reference from the main text to find the exact statements that were used to identify the relevancy of certain concepts. Facilitati
ng services and conditio
ns. Accepta
nce DH central coordinators Ease of use is important. We don’t want to be fiddling around with mobile phones. Entry Account
ability Administ
rative work Verwijtbaarheid Current work process involves a lot of paperwork and back and forth confirmation of information. We believe that IT can contribute a lot to improving these processes, just by alleviating all that paperwork. I am concerned that the officer on patrol gets to do more administrative work since supportive workforce is being decreased All I see and experience is that is that my time behind a desk is becoming increasingly more while I’m less and less on the streets Ease of use 92 DH central supportive team We have to fill in the same thing every time again. It’s a hassle and not user friendly. Rdam central coordinator Local support staff Rdam Officer Dienders centraal Local support staff Entry has to be simple instead of 16 handlings or so. We have to use all these passwords, before you know it the thief is long gone. Make it easy for colleagues to access systems. Heres a manual for the new system. Last time this manual wasn’t even up to date. Agents want drastic changes before they are convinced and willing to work with it. Less administrative work means more blue on the streets. Measuring the impact of information technology systems on police practices Accurate informat
ion Timely informat
ion (Faster,d
irect) More informat
ion Informat
ion retrieval from citizens Receive Faster access of informat
ion Timely, direct Information certainly has added value for us, but especially for the end-­‐user. If an officer can briefly look up information in between tasks, that would be great (indicating the future use of a smartphone with direct access to information) Im afraid that with the lost Accurate An officer knows best when it comes to the information that he has observed… if this info goes from one to another, it may get diluted. If a user can directly input information, then information would reach the database earlier Officers will deliberately choose the wrong form to fill in, because it has less fields to fill People are lazy… they just fill it in carelessly. Now officers just note a few keywords If we would have a laptop or mobile phone now, we could look up information directly When you call the service center you are put on hold, citizens hang up and we miss out on valuable information. IT is also not accessible for people around here, some cant speak Dutch let alone write a mail in Dutch. Now we stopped this Currently, Information travels back and the Measuring the impact of information technology systems on police practices 93 informat
ion (out to date) More informat
ion of knowledge, that more mistakes will be made. The wrong car will be confiscated and wrongly signed as complete Misc. concepts Informatio n overload Readabilit
y Understan
dability Conservati
on of knowledg
e guy again while we just did it yesterday. The information in the system is out to date. information we get from the systems is very poluted It would be convenient if I had a picture…Our computer system only shows hit or no hit, more contextual information would be fine. Now we have to call for extra information. We could use more information to get to know more possible options of completing a task You don’t want to overwhelm the user, we would suggest a possibility to select which items to display. Information should be compact and seen at a glance. We had a moment where we had too many hits on the license registration scanner. This resulted in an overload of information that overwhelmed our capacity Frequent personal contact and knowing what happens on the streets is important. Knowing what happens on the street has to do with knowledge. Most collegues don’t know the codes. There are a lot of codes and I know them. Try to read this part. Do you understand what he wrote there? I think that they want to implement a “Haags allround” model where agents do a little of everything instead of specialized teams. We are afraid that with the shifted focus of execution tasks to officers having execution as a side task that a lot of knowledge will be lost. Execution is not just collecting fines. It is a complex matter. The three of us have been doing this for a while and know every ins and outs about execution. This knowledge can be lost if you try to replace a coordinating team and throw tasks right towards officers on the street. 94 Measuring the impact of information technology systems on police practices forth between cjib, papos and ops. This results in outdated information. Awarenes
s Alternativ
es If people do execution as a side task, they will probably forget each time they have to do it again. Als ik op zon mobieltje ook een foto te zien krijg, dan zal ik eerder doorhebben dat ik de gozer voor mij degene is die ik moet hebben. Quality of service towards public We have to make sure that a police officer has tried out all options possible We could use more information to get to know more possible options of completing a task Profession
alism (authority, image, approacha
ble) Now were in a situation were we show them papers as proof, I don’t know how your image as an officer is portrayed if you show As police officers we send out letters to citizens, this has a sort of threatening vibe so that citizens Recall I can explain the codes to collegues, but they will forget it anyways. As a police organization there is the aspect of providing service towards the public. Sometimes our service towards citizens tends to decrease due to capacity cuts. As an officer you want to portray a certain image. If you are constantly looking at your phone, this does not make you look There must be one unified way of handling so that citizens know what to expect. As a police officer, IT should also help you in provding services towards the public. For example, the system suggesting you how to help a citizen contact other public services. If you are consistent and handle according to a unified way, citizens know what they can expect and that warrants your image and credibility towards citizens. Measuring the impact of information technology systems on police practices 95 Task Innovatio
n Learning them your mobile phone instead. Were not mail delivery men. show themselves. IT should enable people to learn and gain new knowledge. Play as you learn just as in computer games. I am limited in coordinating things more efficient and effective based on authorization issues. From an agents perspective, they sometimes long to be forgiving and flexible towards citizens. Profession
al freedom Prioritizati
on People don’t want to do execution. If … is on vacation, you bet that when he’s back we have to start all over again. Consistenc
y (precision) An element of coordination of officers is that sometimes we want officers to do patrols for burglary, in such circumstances we do not want execution notifications to come through so that these officers prioritize something else. As an officer you are there for the citizen. If you see a woman with 5 kids, you need the room to make a choice not to take in the woman and leave the kids there. Instead you would want to arrange a payment plan because you know that she will pay you eventually. As a police organization you have to be consistent towards the citizens. 96 Measuring the impact of information technology systems on police practices up approachable. Integrity If we are shifting from 8 to 3500 officers who can make changes in the system. Officers could proceed in favor of sympathy to make exceptions and theres no one to check them because they can turf the list on their own. I’m also wondering who’s going to provide authority and control? Im not sure if a smartphone can take over? I am the one who needs to check whether things are filled in correctly. Coordinati
on I’m also wondering who’s going to divide and coordinate the tasks? Productivi
ty Turnaround time Efficiency Less execution tasks being accomplished can be the result of automatisation If tasks arrive on a mobile phone, and then? People don’t want to do execution. Somebody has to be chasing them to do their work and a mobile phone is not going to do that. When an execution task is accomplished, someone has to control whether the task is executed correctly We are wondering how the aspect of coordination is considered with the future introduction of OPP Accuracy of decision It is possible that due to less knowledge, an officer will confiscate the wrong You want to maximize utilization of your resources. Thus you don’t want an agent to visit someone again because he didn’t do it right the first time. Supervisio
n/control Measuring the impact of information technology systems on police practices 97 Fouten maken car and thus make wrong decisions based on lack of knowledge. Confidenc
e Safety Loss of data risk Personal approach Sense of responsibi
lity During the presentation of dienders central, a police agent sketched a situation of how information led policing has made him more confident at taking decisions involving tracking criminals. We can imagine that an officer is busy and distracted staring at his screen and would get stabbed in the neck for a fine of a few euros. Not a concern Safety can only get worse. If I leave my cellphone in the car, before you know it someone has access to all the information of the police. Now we keep things personal by personally visiting the districts and handing the tasks personally. How we approach things is to do things digitally but still maintain a personal touch. Frequent personal contact and knowing what happens on the streets is important. I am the one who personally contacts the agents, I also give them something to compare with. If other agents have 5 tasks, how can it be that you only have 3? Now everything feels as a mandatory thing. There’s no voluntariness in acknowledging the importance of inputting information. IT should trigger people to exchange information with each other. Commun
ication A personal and positive approach motivates officers to finish their tasks (creates responsibility) Cultural conflicts Gap operatio
Im concerned that the involvement of users I feel that IT is just something that is here. The IT guys think 98 Measuring the impact of information technology systems on police practices Heres a service module, do whatever with it. nal vs it Decentra
lization of power DH supportive staff: Now theres sort of separation of power, if a task is not executed properly, who will be there to tell them that its not allowed (when they are allowed to check items at completed themselves) is not considered during development. that everything will be better by using it. Measuring the impact of information technology systems on police practices 99 APPENDIX B: VERIFICATION CHECKLIST The table below gives an overview of the sources of the labels used for the concepts identified in chapter five that discusses the 42 identified concepts. Literature Trkz
Sedera dh Delone Entry Accurate information Timely information Complete information Receive Accurate information Timely information Complete information Other concepts Recall/retain Alternatives Information overload Understandability þ þ þ Management control Service towards public Task innovation Productivity Accuracy of decisions Learning Administrative work Professionalism Personal approach Communication Confidence Consistency Safety Security Cultural conflicts Awareness Response to crime Ease of use Facilitating conditions Acceptance/attitude Frequency of use Time use Stress Information from citizens Readability þ þ þ þ þ þ þ þ þ þ þ þ þ Knowledge Responsibility Integrity Sense of responsibility Efficiency Prioritization Pro-­‐active policing þ
þ
þ 100 Chan (200
1) Groff (2008) Misc. Empirical confirmation Interviews Userstories þ
þ
þ
þ
þ
þ
þþþþ þ þ þþþþ þþ þþ þ
Mason (1978) þ
þ
þ þ
þ þ
þ
þ
þ
þ
þ
þ
þ
þ
þ
þ
þ
þ
þ
þ
þ
þ
Mason (1978)
þ þþ þ
þþþ þ
þ þ þ þþþ
þþ þþ þþ þ þ
þþþþ þþþþ þþþþ þ þ þ þþ þ þþþ þ
þ
þþþþþ
þ
þ
þ
þ
þ
þ
þ þ þ þ þ þ þ þ þ
þþþþ
þ
þ
þþ
þ
þþ
þ þ þ
Measuring the impact of information technology systems on police practices APPENDIX C: SORTING CARDS USED IN SORTING SESSIONS Uitvoer van accurate
informatie
De mate waarin IT mij voorziet
van accurate informatie uit
systemen
correcte informatie, informatie
zonder fouten
Uitvoer van tijdige informatie
De mate waarin IT mij voorziet
van tijdige informatie in
systemen
op tijd, zo snel mogelijk
Uitvoer van complete
informatie
De mate waarin IT mij voorziet
van complete informatie in
systemen
voldoende informatie, meer
informatie
Persoonlijke benadering
De mate waarin IT mij
ondersteunt in het persoonlijk
benaderen van collega’s en
burgers
Communicatie
De mate waarin IT mij
ondersteunt in het
communiceren en uitwisselen
van informatie
Invoer van accurate
informatie
De mate waarin IT mij
ondersteunt in het invoeren
van accurate informatie in
systemen
correcte informatie, informatie
zonder fouten
Invoer van tijdige informatie
De mate waarin IT mij
ondersteunt in het tijdig
invoeren van informatie in
systemen
op tijd, zo snel mogelijk
Invoer van complete
informatie
De mate waarin IT mij
ondersteunt in het invoeren van
complete informatie in
systemen
voldoende informatie, meer
informatie
Leesbaarheid
De mate waarin IT zorgt voor
leesbare informatie
Kennisbehoud
De mate waarin IT ervoor zorgt
dat kennis behouden blijft
impliciete kennis, specialistische
kennis, vakskennis
Overvloed aan informatie
De mate waarin IT voorkomt/er
toe draagt dat ik overspoeld
wordt door informatie
Management en controle
De mate waarin IT mij
ondersteunt om mijzelf en/of
mijn team aan te sturen en te
controleren
teveel informatie
Begrijpbaarheid
De mate waarin IT mij
ondersteunt in het begrijpen
van informatie
verstaan, bevatten
Vasthouden van informatie
De mate waarin IT mij
ondersteunt in het bijhouden
en onthouden van informatie
terugroepen, herroepen
Service naar de burger
De mate waarin IT mij
ondersteunt in het verlenen van
service naar de burger
Taak innovatie
De mate waarin IT mij het
mogelijk maakt om nieuwe
dingen te doen die voorheen
niet mogelijk waren
De mate waarin IT mij discretie
in handelen geeft
professionele vrijheid
Alternatieven
De mate waarin IT mij
ondersteunt in het ontdekken
van alternatieve
mogelijkheden om taken uit te
voeren
Productiviteit
De mate waarin IT mij
ondersteunt in het voltooien
van taken
meer voltooide taken
pop up meldingen, suggesties
Verantwoordelijkheid
De mate waarin IT ervoor zorgt
dat ik meer
verantwoordelijkheden krijg
Accuraatheid van uitvoering
taak/beslissing
De mate waarin IT mij
ondersteunt in het maken van
de juiste keuze bij het
uitvoeren van taken
minder fouten
Zelfvertrouwen
De mate waarin IT mij meer
zelfvertrouwen geeft in het
uitvoeren van taken of maken
van beslissingen
zekerheid, beter gevoel
Consistentie
De mate waarin IT het mij
mogelijk maakt om consistent
taken te verrichten of
beslissingen te maken
Integriteit
De mate waarin IT mij
ondersteunt in het integer
handelen
Besef
De mate waarin IT ervoor zorgt
dat er besef is over het belang
van informatie
Facilitaire condities
De mate waarin ik ondersteunt
word in het gebruik van IT
Acceptatie
De mate waarin ik bereid ben
om gebruik te maken van IT
Frequentie van gebruik
De hoeveelheid IT ik gebruik
Tijdsgebruik
De mate waarin IT het mogelijk
maakt om sneller taken af te
handelen of uit te voeren
Situatie bewustzijn
De mate waarin IT mij
bewuster maakt van de
situatie om mij heen
Informatie van de burger
De mate waarin IT ondersteunt
in het inwinnen van informatie
uit burgers
Professionaliteit
De mate waarin IT invloed heeft
op mijn professionele
uitstraling
Credibiliteit, authoriteit
Veiligheid
De mate waarin IT invloed heeft
op mijn veiligheid
Risico
De mate waarin IT invloed heeft
op het risico op verlies of
diefstal van data of informatie
Respons op criminaliteit
De mate waarin IT mij
ondersteunt in het reageren/
bestrijden op criminaliteit/
misdaad
Culturele conflicten
De mate waarin IT ervoor zorgt
dat er een kloof ontstaat
operationele en IT collegas
Gebruiksgemak
De mate waarin IT makkelijk in
het gebruik is
Proactief politiewerk
De mate waarin IT het mij
mogelijk maakt om pro-actief
politie werk verrichten
Leren
De mate waarin IT mij
ondersteunt in het leren van
nieuwe kennis
Administratief werk
De mate waarin IT de
hoeveelheid administratief
werk beïnvloedt
Efficientie
De mate waarin IT ervoor zorgt
dat ik met minder moeite/
resources meer gedaan kan
krijgen
Prioritering
De mate waarin IT mij
ondersteunt in het prioriteren
van taken van mezelf en/of
mijn team
Stress
De mate waarin IT zorgt voor
emotionele spanning of
stress
Measuring the impact of information technology systems on police practices 101 APPENDIX D: DIFFERENCES BETWEEN OPERATIVE OFFICERS AND SUPERVISORS These data visualizations are derived from the card sorting data and built using the R and the ggplot2 package. 102 Measuring the impact of information technology systems on police practices Concept p.value Taak_innovatie Difference between means 2.801587302 0.005343334 Mann–Whitney U test Significant Management_controle 2.579365079 0.054646207 Acceptatie 2.531746032 0.027679158 Significant Alternatives 2.412698413 0.079643285 Prioritering 2.333333333 0.018768556 Significant Productiviteit 2.071428571 0.031095378 Significant Efficientie 1.976190476 0.364400415 Proactief 1.634920635 0.054236105 Professionaliteit 1.523809524 0.283432195 Situatie_bewustzijn 1.5 0.105078462 Stress 1.246031746 0.352606328 Service_naar_burger 1.238095238 0.271784942 Gebruiksgemak 1.142857143 0.198004108 Overvloed_info 1.023809524 0.425647385 Faclitaire_condities 1.015873016 0.311085531 Invoer_tijdige_info 1.015873016 0.369753676 Recall 1.015873016 0.337177568 Tijdsgebruik 1.007936508 0.140987493 Besef 0.992063492 0.519267426 Integriteit 0.952380952 0.536931071 Verantwoordelijkheid 0.952380952 0.539917502 Frequentie_van_gebruik 0.825396825 0.374602861 Leesbaarheid 0.76984127 0.598446092 Communicatie 0.753968254 0.601218477 Zelfvertrouwen 0.746031746 0.482950351 Risico 0.69047619 0.669107056 Veiligheid 0.666666667 0.714018949 Culturele_conlficten 0.658730159 0.708565912 Begrijpbaarheid 0.626984127 0.460244648 Uitvoer_complete_info 0.611111111 0.8701055 Accuraatheid_beslissingen 0.563492063 0.40527349 Leren 0.515873016 0.668496304 Invoer_complete_info 0.5 0.57509752 Invoer_accurate_info 0.492063492 0.850106739 Persoonlijke_benadering 0.468253968 0.601362812 Uitvoer_tijdige_info 0.293650794 0.825175093 Respons_criminaliteit 0.253968254 0.80552976 Informatie_van_de_burger 0.246031746 0.781801919 Consistentie 0.222222222 0.975444397 Kennisbehoud 0.063492063 0.975281615 Uitvoer_accurate_info 0.023809524 0.515446864 Administratief_werk 0.023809524 0.681636195 Measuring the impact of information technology systems on police practices 103 APPENDIX E: DISTRIBUTIONS OF CARD SORTING DATA The histogram shows that the data is not normally distributed, what the data does show however is tendencies towards certain values. To increase the visibility of these tendencies, we compiled separate histograms for the perspective of supervisors (blue) and operative users (red) each. 104 Measuring the impact of information technology systems on police practices Measuring the impact of information technology systems on police practices 105 APPENDIX F: LITERATURE REVIEW As discussed in the research design and system analysis chapter a literature review was conducted for the theoretical analysis. This appendix gives a detailed explanation of the process that led to the identification of two suitable sources for use as a theoretical anchorpoint for the mental map construction. For the systematic literature review on the topic of impact we started the review by searching Google, Google scholar, Scopus, Web of Science, IEEE and Jstor on single or combinations of keywords such as information system(s), IS, information technology impact, acceptance, and technology. We also searched for keywords relating to the word impact as a topic. These words ranged from performance, effectiveness, consequences of use and effects on job or tasks. We extended the search with SOA, success, police and law enforcement as additional keywords. The inclusion of success as a keyword was because authors that refer to IS-­‐success literature were most reflective and explicit about different aspects of impact. The inclusion of SOA (or web-­‐based, cloud) was also considered since the information system developed in this case study was developed following SOA architecture. We later decided to exclude SOA because properties of SOA are very indirectly related to the definition of impact we considered within our scope (motivation for scoping is discussed in chapter 3.). We also extended the use of keywords to keywords derived from internal references in the articles that were found. This resulted in 51 found articles of which 12 articles were related to policing or law enforcement contexts. The table below contains an organized recap of the insights derived from the systematic literature review. The first column contains the author, the second column contains brief keywords on the constructs that were the topic of focus in the article, the third column contains a check if it consisted of research in a police context, the fourth column describes the underlying assumption of the research (deterministic: focused on researching causal relationships between constructs, emergent: focused on researching the contextual variety of construct), the fifth column indicates whether the author provides explanation for the chosen constructs or measures, the sixth colum contains a brief description of the study, the last column indicates whether the article is focused on aspects that indicate consequence of use ( we considered consequence of use to be important based on the level of impact analysis chosen in chapter 2.) Systematic review of these papers resulted in the following findings: •
•
What is notable is that 19 of the 51 found articles only refer to consequence of use or impact as a line of argumentation (last column in table). These articles do not explicitly define consequences of use let alone provide concepts or frameworks for measurement of consequences of use. Most of these studies that do mention impact or consequence of use are usually focused on the topic of acceptance or technology fit. These researches all rely on the assumption that acceptance and technology fit are important predictors for use and in its place impact, therefore they focus on identifying and perfecting predictors for use and instead of identifying concepts of consequences of use itself. This leaves us with 32 of the 51 found articles that explicitly mention and define terms or discuss topic relating to consequences of use. What is striking is that only nine of the 32 articles explicitly discuss how they derived their constructs of consequences of use (fifth column in table below labeled ‘explanations for measure’). All other articles do not provide line argumentations or explanations for the choice of a certain amount of measures indicating consequences of use. 106 Measuring the impact of information technology systems on police practices •
Out of the nine articles above that discuss consequences of impact, we found that two articles were useful to serve as anchor point in literature for identifying impact constructs. The choice for (Torkzadeh & Doll, 1999) and (Gable et al., 2008) is based on the fact that they provided explanations for their proposition of four constructs representing individual impact. Furthermore, these authors also noted that they have reviewed the available literature and narrowed them down to these four constructs. Measuring the impact of information technology systems on police practices 107 Author Focused concepts Police context? Explanation for chosen measures? No Focus of study Consequences of use? Emergent vs deterministic? Deterministic Abugabah (2009) User performance Developing a causal model Organizational performance and effectiveness Information technology, information system, work system, firm, organization Acceptance Deterministic No Causal relationship between IT and effectiveness and organization Yes, efficiency and effectiveness Yes, organizational and impact on job Argyropoulou (2013) Emergent n/a Describing various views on systems n/a Deterministic No Acceptance of SOA by IT professionals No Success, acceptance Deterministic No No (Briggs, 2001) (Briggs, 2003) (M. Brown, 2014) (Longitudinal) intention to use Productivity Deterministic No Predicting system success by acceptance n/a n/a n/a (S. Brown & Peterson, 1994) (Brynjolfsson & Yang, 1996) (Chan et al., 2001) Effort, Performance, Satisfaction Productivity Deterministic n/a Deterministic No ‘Police practices’ þ n/a
No
(Chuttur, 2009) Acceptance þ Deterministic n/a (Colvin & Goh, 2005) Acceptance þ Deterministic
No
(Cyrus, 1991) Effectiveness Deterministic No (Devaraj & Kohli, 2003) Organizational performance, use Success constructs Deterministic No Yes, only for system and information level, not for Explanation of productivity as a concept being impacted by IT Effects of effort on performance and satisfaction Explanation of productivity as a concept being impacted by IT Exploring how police practices are impacted by IT
Overview of technology acceptance model Acceptance applied and validated in police context
Definiing effectiveness and measurement of it Causal link between use and organizational performance. Proposes measures for system and information level and validated these extended levels in casual relationships in IS-­‐success model Alter (1999) Baumgartner & Green (2011) Behrens (2005) (Daskalakis & Mantas, 2008) No n/a Yes, productivity in financial measures Yes, uncategorized
No No
Yes, organizational and monetized measures No (DeLone & McLean, 1992; Delone & McLean, 2003; Petter, DeLone, & McLean, 2008) (Dishaw & Strong, 1999) Success Emergent Yes Proposes levels of success measures Yes Deterministic No (Fryk, 2007) Acceptance, task-­‐
technology fit Process identification Emergent n/a (Gable et al., 2003, 2008) Success constructs (Galve-­‐Górriz, 2007) Organizational performance (Garicano & Heaton, 2006) Organization of policing, police effectiveness Task tech fit, individual performance n/a þ Deterministic Yes (limited to a statement that they have validated measures but no detailed explanation) Yes, hypotheses are provided with arguments No Focuses on identification of changed information flow in health care Yes, limited to mentioning impact N/A Deterministic No þ n/a (Hekim, Gul, & Akcam, 2013) (H. C. Lucas & Nielsen, 1980) (Hu, Chen, Hu, Larson, & Butierez, 2011) (Heejin Lee & Whitley, 2002) (HS Lee, Kim, & Choi, 2012) (Lind & Lipsky, 1971) Use, outcomes of investigations Presentation, learning, performance Acceptance þ Deterministic Yes, system changes and consequences of changes are motivated with examples No Deterministic No þ Deterministic No n/a n/a n/a Acceptance Deterministic Police output þ (H. Lucas & Spitler, 1999) (Maier, Laumer, Eckhardt, & Weitzel, 2013) Performance Job satisfaction, turn over intention, attitude, (Goodhue & Thompson, 1995) (Groff & Mcewen, 2008) Yes Proposes three hypotheses of IT impact on organizational performance Yes TTF model Agency composition, worker skill, crime rates Perceived impact Focuses on identifying measures per implemented system Yes, uncategorized measures per system Focuses on validating relationship use and outcomes of investigations Presentation influencing learning and performance Measuring acceptance Yes, clearance rates, response rates Yes, learning, performance No Provides illustrative examples per impact level Acceptance of smartphone applications Yes, societal, individual, organizational No Emergent Yes Conceptualizing police output Deterministic Deterministic No Yes Use influencing performance Acceptance on satisfaction and turnover intention (work-­‐related outcomes) Yes, but no operationalization Yes, revenue No No Measuring the impact of information technology systems on police practices 109 (Mason, 1978) usefulness Information output Emergent Yes Information output on various levels (OECD, 2008) n/a n/a Impact of ICT on society from an economic perspective (Olugbode, Elbeltagi, Simmons, & Biss, 2008) Deterministic þ þ n/a Deterministic n/a (Singh & Kroesen, n.d.) Efficiency, integration, strategy, communication, performance, profitability n/a Acceptance, technology task fit Performance and process No, choice motivated by availability data No þ n/a Yes (Tatar, 2011) Individual, organizational n/a n/a (Torkzadeh & Doll, 1999) Individual impact Emergent Yes (UNCTAD, 2011) Society n/a No (Schaik, 1990) Profits n/a n/a (Venkatesh et al., 2003) (Watson & Driver, 1983) Acceptance Graphics on recall Deterministic Deterministic Yes n/a (Yalcinkaya, 2007) (Zhou, Lu, & Wang, 2010) Adoption TTF, UTAUT þ Deterministic Deterministic No No (Pica & Sørensen, 2004) (Saviak, 2007) 110 Measuring the impact of information technology systems on police practices Focuses on casual relationship between several factors on performance and profitability Contexts of police work Predicting use in law enformcent context Qualitative description of impact on law enforcement Argumentation for the importance of measuring individual performance Identifying measures for individual impact United nations research of impact on society Effectiveness in terms of monetary values Acceptance model Casual relationship presentation of info on recall Police officers adoption of IT Integration of two models Yes, but limited to processing of information Yes, Societal economic impacts Yes, but no specified operationalization No No Yes, but limited to description of performance changes ( time, resources, workflows) and process changes No specification Yes Yes, economic measures Yes, monetary values No Yes, recall No No APPENDIX G: CATEGORIZATION OF LEVELS OF IS-­‐
IMPACT ANALYSIS Because of the abundance and ambiguity of impact terminology used by various authors, we developed the following impact level scale containing seven levels. The origins of these seven categories are illustrated in the figure below system creation
Production
Product
system use
Psychological
beliefs
attitude
user satisafaction
Use
Individual
Organisational
behavior
impact on
work on
individual
level
organizational
impact
system usage
change in behavior
receipt
application
product
Society
System to
value chain
( Torkzadeh et.
al., 1999)
General
measures of
effectiveness
( Cyrus, 1991)
- productivity
- cost-benefit analysis
- value analysis
understanding
production
consequences of use
levels of
influence
information
output (Mason,
1978)
change in
organisation
al
performance
(Alter,1996)
information
technology
information
system
system
quality
information
quality
IT artifact
work system
satisfaction
satisfaction
intention
use
use
organisation
firm
IS Success
(Delone &
Mclean, 1992)
net benefits
Banbazat &
Zmud (2003)
impact
individual
output
organization
al output
societal
output
Lee & Whitley
(2002)
Figure 7.4-­‐1 Impact level scale. As the figure shows, many definitions and scopes exist on impact. The figure includes views of seven different authors which has been categorized under seven generic levels which are production, product, psychological, use, individual, organizational and society. The scale is a continuous scale that ranges from system creation levels on the left side to system use and consequences of use to the right side. Measuring the impact of information technology systems on police practices 111 APPENDIX H: DESIGN OF WEIGHTED SCORECARD MEASUREMENT TOOL In chapter six, this research proposed four possible measurement scenarios of which the last measurement scenario focuses on designing a measurement tool based on the highest scoring indicators without consideration of the current state of the belevingsmonitor. This appendix contains a detailed description of how operationalization of indicators in scenario four: a weighted scorecard was designed. OPERATIONALIZATION A measurement tool is not complete without instructions on how to measure the identified concepts. Therefore, this section will present one of the possible ways to operationalize the relevant concepts for use in the context of OPP E&S. The operationalization is motivated by practical reasons as to provide an example or reference for the client of this research. By no means does this research suggest the results presented below to be the best and only way to operationalize measurement. The operationalization has been conducted using the following criteria: •
•
Feasibility. There is no use for evaluating the reliability or validity of a measure if a measure cannot be operationalized a certain way. If the concept itself does not represent an observable or physical phenomenon, the measurement of this concept is inherently bound to be subjective (Elasy & Gaddy, 1998). This limits the use of objective measures such as counting amounts or keeping time. Reliability & efficiency. Prioritize precise and accurate operationalization balanced with the resourced needed. The proposed operationalization for all concepts is in discussed in groups of three. The first operationalized concepts discussed consist of relatively simple concepts that can be expressed in amounts or timespans. The second group consists of concepts that can be classified as behavioral phenomena. The third group consists of concepts relating to concepts of information quality. OPERATIONALIZATION RESULTS The concepts listed in table 0-­‐1 all shares the same characteristic as these concepts can be expressed by amounts. Productivity can be expressed as the amount of entities being produced (Pritchard, 1990). Efficiency can be expressed as the amount of resources (time or effort) needed to complete a single task (Groff & Mcewen, 2008). Ease of use has characteristics that can be expressed by the amount of steps needed to complete a task (Privitera, 2011). Time use can be expressed as the amount of time spend behind an IS (United Nations, 2005). Administrative work can be expressed as the amount of time spends doing administrative tasks in ratio to operative field work (Chan et al., 2001). Measuring concepts by recording or approximating amounts or values provides much richer information compared to the use of ordinal rating scales used in most surveys (Sekaran, 2006). Since 112 Measuring the impact of information technology systems on police practices all these concepts can be expressed in tangible or observable amounts, this research proposes the use of objective measures to operationalize these concepts whenever possible. •
•
•
•
•
Productivity amounts are already measured periodically, thus the amounts can be withdrawn from performance reports. Amount of man-­‐hours spent on tasks can be measured through observing and manually recording the amount of hours one spent. We expect this to be very resource intensive, since an observer needs to follow an individual during a certain period to compensate for fluctuations in an individuals activity and multiple individuals to compensate for differences between individuals Due to expected the resource intensiveness, it is much more affective to use survey approximations and let users themselves approximate how much hours they spent on execution tasks during a month. Since measurement of the amount of steps can be monitored automatically, observation is not necessary. The amount of steps can just be recorded. IS use is very infrequent and also in timespans of minutes, and therefore the use of survey approximations expected that survey approximations will contain too much errors since it is difficult to estimate the infrequent timespans behind an IS. Therefore, sampled observation is necessary The amounts for the ratio administrative/operative work can be derived through survey estimations, following the same motivation provided for the measurement of efficiency, observation would be too resource intensive as an observant would need to be followed for an extensive time. Concept Productivity Unit of measurement Current situation Average amount of execution tasks Performance reports completed per month/year Average turnaround time of a completed execution task cycle Efficiency Man-­‐hours spent on execution-­‐related Survey approximation tasks Ease of use Amount of steps needed from extraction Record of information to a completed overview of an individual execution task Time use Time spent behind IS to input or output Observation information. Administrative work Average ratio time spent on administrative Performance reports work / operative work for execution tasks Table 0-­‐1 Operationalisaiton of concepts that have ratio characteristics After implementation Performance reports Survey approximation Record Observation Survey approximation Proactivity, task innovation, management control, communication, acceptance and preservation of knowledge are concepts that represent social behavioral phenomena. Behavioral phenomena do not occur physically and therefore, the measurement of behavioral phenomena is inherently bound to subjectivity. Considering this, this research proposes to measure proactivity, task innovation, management control, communication, acceptance and preservation of knowledge should therefore be measured through surveys, since surveys are the most resource efficient in performing subjective measurement. Concept Proactivity Task innovation Management control Communication Acceptance Unit of measurement The extent that IT assists me in performing tasks more proactively The extent that IT assists me in trying out new ideas in performing tasks The extent that IT assists me in managing and controlling myself and my team The extent that IT assists me in communicating in my team The extent that IT is accepted by me or my Current situation Survey ratings After implementation Survey ratings Survey ratings Survey ratings Survey ratings Survey ratings Survey ratings Survey ratings Survey ratings Survey ratings Measuring the impact of information technology systems on police practices 113 team The extent that IT preserves or assists in the preservation or improvement of explicit and implicit policing knowledge and expertise Table 0-­‐2 Operationalisation of behavioral related concepts Preservation knowledge of Survey ratings Survey ratings Accuracy, completeness and timeliness are all dimensions of information quality (Wang & Strong, 1996). Although information quality refers to much more than characteristics of information itself, the information quality concepts identified in this research are limited to simple concepts, which are expressible in amounts. The amount of reported faults, amounts of fields displayed and age of information can be automatically derived from the system itself. In situations where these characteristics are not stored alongside the data, operationalization can be done by means of random sampling. Concept Input of accurate information Output of accurate information Output of complete information Output of timely information Unit of measurement Reported faults Current situation Random sampling After implementation Record Reported faults Random sampling Record Average amount of fields displayed Random sampling Record Random sampling Record Average age of information (compared to date of access) (Wang and Wand 1996) Table 0-­‐3 Operationalisation of information quality related concepts 114 Measuring the impact of information technology systems on police practices APPENDIX I: INTERVIEWS This appendix contains the documentation of the interviews documented following to the claims, concerns & issues methodology. This means that documentation was limited to statements or narratives that were identified to be a claim, concern or issue. Each narrative or statement documented below was given a label either derived from literature (appendix B) or from a keyword within the narrative or statement. For a detailed description of the methodology and sample please consider section 3.3.2. For an overview of the statements categorized per concept please consider appendix A. The interviews are un-­‐translated. Claims, Concerns & Issues (Koch 1994) Claims A claim is any assertion that a stakeholder may make that is favorable and is a platform from where the team can take the next steps. For example it might be that the staff feel they already have good teamwork on the ward.. Concerns A concern is any assertion that is unfavorable. The purpose is to highlight potential barriers: personal, systematic or organizational, real or perceived. An example might be that a collaborative nursing model will require a change to the way the rosters are usually structured. Issues An issue is a reasonable question raised through better understanding of claims and concerns. Issues are drawn from claims and concerns using What and How questions and issues will need to be addressed before progressing with the project. DOCUMENTATION OF INTERVIEWS Administratieve werkers Deze twee administratieve werkers ondersteunen het executieteam van Den Haag zowel in administratieve handelingen maar ook het coördineren, assisteren en informeren van operatieve politie agenten. Administratieve taken Ze is ervan bewust dat het huidig werkproces buitengewoon omslachtig is. Veel papierwerk, veel heen en weer terugkoppeling van informatie. IT kan al heel veel betekenen al is het maar dat al dat heen en weer gedoe en al dat papierwerk plus mappen verminderd worden. Die dingen kunnen digitaal. Gebruiksvriendelijk (Voorbeeld van hoe betalingen verwerkt worden werd omschreven door medewerkers). Agenten en administratieve medewerkers moeten veelvoudig documenten invullen ( fysiek en digital). Alles moet steeds opnieuw ingevuld worden. Het is een gedoe en niet gebruiksvriendelijk. Als het nou allemaal digitaal wordt en al voor gevuld wordt dan scheelt dat een hoop. Ze noemt ook een voorbeeld waarbij dingen afgeturfd moeten worden. Officieel mogen zij zaken in OPS niet afturven. Daarvoor moet een hkd-­‐er eigenlijk gebeld worden. Maar dit is zon administratieve rompslomp dat zij het zelf maar doet. Informatie Het belang van informatie: Administratieve medewerker ziet zeker de toegevoegde waarde van een mobieltje bij de eindgebruiker. “informatie is heel belangrijk”. Administratieve medewerker ziet zeker het belang van informatie dat direct toegankelijk is voor de agent. Als ze zelf even snel iets op kunnen zoeken dan is dat zeker fijn. Decentralisatie Volgens mij willen ze naar een Haags model gaan waar elk agent een beetje van alles en nog wat oppakt. Hiermee zorg je er eigenlijk ook voor dat centraal kennis naar gedecentraliseerde kennis gaat. Measuring the impact of information technology systems on police practices 115 Kennis en kwaliteit van informatie: Wij zijn hier gespecialiseerd en hebben kennis van zaken. Als agenten straks dit gaan doen als erbij. Nu kunnen ze naar ons bellen als iets niet duidelijk is. Moeten ze het straks maar uitzoeken? De kwaliteit van informatie en kennis gaat daarmee achteruit. Fouten in informatie Meer fouten, agenten nemen dan een voertuig in beslag terwijl ze die voertuig helemaal niet in beslag mogen nemen. Agenten vullen daarna ook nog eens het formuliertje verkeerd in en turven onterecht een opdracht als compleet af. Integriteitsrisico: we gaan van 8 mensen naar 35000 mensen die dan wijzigingen mogen aanbrengen. Op dit moment zit er nog een (psychologisch) controle effect dat agenten terugkoppeling moeten geven aan ons. Maar als agenten straks zelf executietaken kunnen afturven dan kan dat wat betekenen voor de integriteit. Ik wil niet zeggen dat dat nu veel gebeurt, maar het is wel een risico. Decentralisatie van macht (integriteit/fouten in beslissingen) Nu is er toch een soort scheiding van machten. Wij dringen erop dat executie taken op een bepaalde manier afgehandeld worden. Als een agent straks een zielig verhaal van burger aanhoort en sympathie krijgt en geneigd is om een betalingsregeling aft e spreken. Wie is er da nom erop te hameren dat het niet mag? Productie Ik denk dan dat de kwaliteit/effectiviteit van taken gaat achteruit. Al met al zullen dan minder executietaken voltooid Controle coördinatie en gezag Waar ik ook afvraagt is: Wie stuurt aan? Wie verdeelt de taken? Wie oefent gezag uit? Coördinatoren executieteam den haag Ik sprak met 3 mensen uit het executie team den haag. Zij hebben voornamelijk een centraal coördinerende rol binnen de regio den haag. Zij brengen de taaklijsten langs bij de verschillende wijkbureaus langs. Zij voeren zelf geregeld ook executietaken uit. Denk bijvoorbeeld om executietaken uit te voeren bij grootschalige verkeerscontroles of toch ook het overnemen van executietaken van wijkbureaus. Voorheen waren zij onderdeel van de parket politie Claims Zij zien ongetwijfeld de toegevoegde waarde van informatie, mobile toegankelijke informatie dat zich uit in ondersteuning. Met nadruk op ondersteuning. Ik zou het fijn vinden alvorens op pad te gaan een foto te kunnen zien van degene die ik moet pakken. Ik weet dan dat degene die voor mij staat niet degene is die mee moet, of dat degene waarschijnlijk de broer is o.i.d. Nu hebben we een mobile system in ons auto. We voeren daar gegevens in ( kenteken uit camera) is en het system geeft hit of geen hit aan. Zou fijn zijn als er meer contextuele informatie mee kan in de vorm van een laptop of mobile. Waarom moet ie mee? Wat is de redden wat moet er precies gebeuren. Nu bellen we naar een centraal nummer wat er precies mee verder moet. Houding van burgers (veiligheid) Wat gebeurt er als je aanbelt? Het is echt niet zo dat ze dan een raampje op de eerste verdieping openen en een bak met geld naar beneden strooien. Je kan niet een mobieltje ertegen aan gooien en daarmee verwachten dat het goed komt. Houding van agenten t.o.v. executie Executietaken worden ervaren als vervelend en kennen geen prioriteit. Vaak zijn het de jonge agenten die een stapel executies meekrijgen, zoek het maar uit. Complexiteit van executie (moeilijke beslissingen) Bovendien is executie niet een kwestie van een boete innen. De ene keer moet je met wet mulder rekening houden en de andere keer met een andere wetgeving. Voorbeeld: Lees maar eens wat hier staat. Denk je dat zon agent op straat enig idee heeft wat hier staat. Wij zijn hier dag in en dag uit mee bezig dat we er enigszins nog iets uit op kunnen maken, maar als je dan zon jong aspirant heb zal het nog wat worden als je die executieopdrachten rechtstreeks naar de agent toe gooit. Verdeling en behoud van kennis 116 Measuring the impact of information technology systems on police practices Kijk nu even naar zon executietaak. Snap je wat er staat? Duidend om de omschrijving: die heel onduidelijk is. Wij kunnen uit ons ervaring nog duiden wat er ongeveer staat. Als een agent zon blaadje voor zich ziet zal het zeker weten een mysterie zijn, en al helemaal straks als het automatisch binnenkomt op zijn scherm, alsnog snapt hij er niks van. Uitgaande van deze context zijn de concerns het volgende: Wat voor beeld hebben jullie daar bij vtspn? Het is namelijk geen kwestie van, we gooien er een mobieltje tegen aan en het komt goed. Hoe gaat het dan met dat mobieltje? Veiligheid We weten niet hoe dat straks in het werk gaat met zon apparaat. Maar straks zit je daar te priegelen op je mobiel: ik scroll even en zoek je gegevens op, kijk even op het scherm meneer, dit moet u gaan betalen. Voor je het weet heb je voor een paar tientjes een mes in je nek. Ze wijzen indirect op de duiding van dat papierwerk nu impliciet voor impact heeft: dat je niet hoeft te priegelen op zon apparaat. Dit is waarmee agenten langs de deur gaan (wijzend op de papieren die letterlijk mee gaan met de agenten, met notities ernaast) Gebruiksgemak > veiligheid Uitstralen van gezag en professionaliteit Wat straalt zon mobieltje uit? Nu is het toch dat je aan de deur komt en laat zien hier: dit zijn de papieren waarop staat dat je moet betalen. Ik weet niet hoe het eruit gaat zien met zon mobieltje. Als je zon mobieltje heb en een lijst door moet scrollen voordat je de taak heb en even je schermpje laat zien. (impliciet duidend op: zon schermpje laten zien is toch minder krachtig dan papier). Het is toch iets anders dan een pakketje bezorgen. Aansturing Als straks coördinatie automatisch op het mobieltje komt van een agent… en dan? De mensen willen geen executie doen. Als je dat automatiseert, wie zit er dan achter hun broek aan. Hij ( duidend op een coördinator) is straks op vakantie voor een maand. Als hij terug komt zullen zeker al die taken stil komen te zitten. Ook het persoonlijk aspect: Wij komen fysiek langs om die papieren te brengen, spreken die jongens persoonlijk enzovoorts. Ze kunnen bij ons terecht voor vragen en onduidelijkheden Automatische coördinatie: In eerste instantie leuk dat een agent direct een executietaak binnen krijgt, maar wat ie verder ermee moet? Duidend op de kwaliteit van coördinatie, zal nooit automatisch kunnen of beperkt. Executie draait eigenlijk om een paar fanatiekelingen (zoals ons) die de waarde van executie kennen. Niemand heeft zin derin, als mensen het ook tussendoor doen, ze zullen zeker weten weer vergeten als ze weer met andere dingen bezig zijn. Dagje meelopen executie verkeerscontrole Direct toegang tot informatie Duidend op het gebrek aan raadpleegbronnen in de auto. Kijk nu moeten we bellen naar een centrale. Nu zou zon laptopje of mobieltje handig zijn. Een mobiele printer zou ook handig zijn, nu moeten we naar het bureau om documenten te printen. Up to date informatie Duidend op een Mercedes die weer aan de kant gezet wordt terwijl die gister al is gecheckt bij een ander verkeerscontrole: Nu houden we dezelfde man als gister aan. Dat laat al zien dat de info in het systeem verouderd is. Beperktheid informatie Verschillende mensen hebben een eigen informatiebron terwijl we allemaal voor dezelfde baas werken. Veiligheid Een snelle afhandeling is belangrijk, nu staat die man daar langs zijn auto en die staat te broeien. Hop, meneer uw auto wordt in beslag genomen, klaar. Nee nu loopt die man ( agent verantwoordelijk voor beslagname) weg en die man staat daar maar. Ik ben er maar bij gaan staan voor de zekerheid bij dat Measuring the impact of information technology systems on police practices 117 kleine meissie. Verantwoording / onzekerheid Die agent weet dat die gozer gearresteerd moet worden, in plaats daarvan loopt hij naar ons busje toe. Vraagt daar nog eens of het in het systeem staat. Jij hebt hem aangehouden dus jij moet hem meenemen. Het zou onzekerheid kunnen zijn, maar kan ook gewoon afschuiving van verantwoording zijn. Gesprek met ondersteuner Den Haag Schilderswijk Gap operational vs it Belangrijk concern die zij aan het begin aankaartte was dat ze zich zorgen maakt om inspraak van de gebruikers bij de ontwikkeling van de systemen. Ze vreest dat er straks weer een system komt die van bovenaf komt, en dat dan bij gebruik men erachter komt dat het systeem tekort schiet. Dan is het al te laat of duurt het veelst te lang voordat de verbetering erin is verwerkt. Administratief werk Een ander belangrijk concern is dat de agent straks meer moet doen. Op de wijkbureau valt straks de info organisatie weg ( die wordt centraal), de administratie moet weg. Hoe verder? Dienstverlening naar de burger / tevredenheid/ maar ook informatiewinning va. Burger Als politie heb je ook het aspect dienstverlening naar de burger toe. Als voorbeeld: Regionaal service centrum: Bedacht voor agenten en burgers om te bellen. Vanuit de burger: 4 kaarten invullen, je staat in de wacht. Burger haakt dan toch af. Zo missen we toch belangrijke intel, potentieel belangrijk info vanuit de buurt lopen we zo mis. We faciliteren ook wel in de mogelijkheid voor binnen lopers. We merken ook meer binnen lopers omdat ze bij de telefoon zolang aan de wacht moeten staan. Maar nog steeds is binnenlopen niet DE oplossing. Nog steeds lopen we potentiele informanten mis ( laat maar zitten met dat bellen, ik ga niet naar binnen). Ons bureau is ook s’avonds niet meer geopend, potentiele informanten die wellicht s’avonds langs willen komen zodat de buren ze niet zien, dat kan nu niet meer. IT id dienstverlening voor de burger: Benaderbaarheid burger Natuurlijk is 24 uur open zijn en altijd binnenlopen ook niet de oplossing. Ik moedig mensen ook aan om ons digitaal te benaderen, via email en dergelijke. Maar bij it moet je rekening houden met de buurt hier. Je hebt te maken met analfabete mensen, of allochtonen mensen die de taal niet spreken. Op een of ander manier moet je die ook kunnen benaderen. Dienstverlening Puntjes/issues: Burger niet gelijk kunnen helpen: als een burger langskomt moeten zij afspraken maken ipv. gelijk kunnen vertellen wat er is. We kunnen de burger niet bedienen, wat te maken heeft met reducering in capaciteit natuurlijk) Invoeren van dingen: Het moet simpeler i.p.v. 16 kaarten invoeren Scheiden van direct invoeren vs later invoeren ( door ondersteunend ) We hebben geëxperimenteerd met een front en back office idee, dat ging niet goed, voornaamste oorzaak is dan toch dat de mensen in de back office vaak oud brigadiers zijn. Dat zijn juist de mensen die niet goed weten hoe ze het in systemen moeten invoeren. Direct invoeren Als iemand het direct invoert en ook zelf doet, is een voordeel dat hij als geen ander weet hoe dingen in elkaar zit. Nadeel is dat je niet direct de tijd of ruimte hebt om alles direct in te voeren, je noteert toch paar steekwoorden en gaat later aan het werk. Later invoeren Door zichzelf: Achter bureau een aanhouding dan uitwerken. Nadeel is dat dit tijd inneemt. Dan duurt het ook een tijdje voordat het in het systeem staat. Door ondersteunend personeel: Informatie overdracht van 1 naar ander kan info verloren gaan of vertroebelen. 118 Measuring the impact of information technology systems on police practices Bijv. Bij aanhoudingen moet een agent toch nog 1 uur binnen zitten. Hoe komt dit? Deels door de processen maar ook gewoon de inefficiëntie. JE zal altijd processen blijven behouden, maar een deel van het papierwerk, daar zouden voor een deel wel verbeteringen voor gemaakt worden. Kern boodschap: We willen meer blauw op straat, maar tegelijkertijd komt er minder blauw op straat doordat er meer taken voor rekening van de agent komen Minder ondersteunend personeel, die in eerste instantie daar waren om die taken van de agent over te nemen. Waar wordt de telefoon nu voor gebruikt. Social media, twitter enz. Worden gebruikt om info uit te geven maar ook inwinnen. Social media is ook fijn om anonimiteit van informant te waarborgen. Beeldvorming/image Als agent wil je een bepaald beeld uitstralen, als je als agent alleen maar op je telefoon zit te kijken, dan zie je op dat moment niet je omgeving. Verder nodigt dat niet uit voor een burger om je te kunnen benaderen. Centralisatie Ik ben voor centralisatie maar: Warm contact moet behouden blijven, het persoonlijke, en weten wat er op straat gebeurt. Weten wat op straat gebeurt is ook beetje het kennis aspect. Weten hoe je van informatie kennis maakt. Het sociale aspect Als agent ben je er voor de burger. Ruimte voor agent om keuzes te kunnen maken: Als je vrouw ziet met 5 kinderen, dat je de ruimte krijgt om te kunnen bepalen wat een betere keuze is. Als je die vrouw meeneemt tbv. Innen van executie, hoe zit het dan met de zorg van die kinderen. Soms voel je als agent al aan dat je de executie taak even uit wil stellen van papos naar ops en is die vrouw dan ook in staat om het wel te betalen na die tijd. Dit hebben we overigens ook aangekaart bij het cjib. Vooraan de keten de mogelijkheid voor een betalingsregeling bieden. Dat is iets wat ik en ook de rechters hebben aangevochten. Die betalingsregeling komt er dan toch, maar hoe verder het proces verloopt hoe meer kosten je eig. maakt als overheids instantie. Als er vooraan in de keten een betalingsregeling geregeld kan worden dan bespaart dat de burger de toenemende kosten en de ketenpartners de capaciteit die vrijgemaakt moet worden om hier werk van te maken. Politie rotterdam feyenoord Sturing Een van de concerns die zij in Rotterdam is in hoeverre er rekening wordt gehouden met sturing. Automatisering is fijn maar die opdrachten moeten ook uitgevoerd worden. Persoonlijke aspect van sturing Hoe wij het nu aanpakken is dat wij volledig digitaal werken met geautomatiseerde excel lijstjes. Wij benaderen de uitvoering van executie opdrachten door de agenten zelf direct persoonlijke mailtjes te sturen. Gevoel van verantwoording Door de persoonlijke benadering maar ook positieve benadering ( het is een soort spelletje geworden), zorgen we ervoor dat die agenten ook die taken afwerken ( he, weer een mailtje van die Ron). Prioritering van uitvoering opdrachten Een ander element van sturing is de mate waarin agenten prioriteit geven of moeten geven aan bepaalde taken. Soms sturen wij agenten op pad om te patrouilleren op inbraak, dan is het soms niet gewenst dat ze dan tussendoor executie opdrachten uitvoeren omdat er een pushnotificatie op zijn telefoon komt. Zelf prioritering vs centrale prioritering Centrale team van executie vs allrounder die af en toe een executie taak oppakt. Wij vinden dat allround afronden van executie beter is want: Als 1 team gespecialiseerd is in executie Measuring the impact of information technology systems on police practices 119 zou je kunnen beargumenteren dat zij meer kennis hebben en er voortdurend mee bezig zijn. Meer kennis is geen argument want executie is niet zo moeilijk. En het voortdurend bezig zijn is ook geen argument aangezien ( hier in R’dam) executie zelfs voor de parketpolitie hier geen prioriteitsnadruk had. Verantwoordelijkheid Wij zijn juist voor de allround agent die executie taken moet afronden. Wij geloven dat als je een boete uitschrijf dat je de volle cirkel moet maken. En dat laatste stuk is ook het innen van de boete. Jij hebt die boete geïnd dus jij bent ook verantwoordelijk voor de executie daarvan. Als je het niet goed afrondt dan komt het hoe dan ook altijd bij jou terecht dus zorg maar goed dat het goed afgerond wordt. Controle Bij het afsignaleren heb je te maken met een positieve en negatieve afsignalering. Een positieve afsignalering is meestal gewoon goed, want dan is er geld geïnd, iemand meegenomen of een auto in beslag genomen. Beoordelen of het werk wel goed gedaan is Maar bij negatieve afsignalering is het toch handig dat Ron komt beoordelen in hoeverre deze negatieve afsignalering wel terecht is. Hebben ze alle mogelijkheden wel geprobeerd? Misschien weten ze niet dat er nog andere mogelijkheden zijn of soms een kwestie van luiheid, dan sturen we ze alsnog op pad. Benadering van burgers De effectiviteit zit er ook in doordat wij een hoop executieopdrachten al afgerond krijgen door het versturen van brieven. Dat zorgt er al voor dat we voor een groot aantal executies niet de deur uit hoeven. Wij sturen brieven uit naar burgers uit namens de wijkbureau in de buurt. Hiermee straal je wel soort dreiging of vorm van gezag uit ( owjee straks komt politie voor de deur) waardoor je zorgt dat ze gelijk verschijnen. Doorlooptijd van executie opdrachten Belangrijk element voor het afronden van de taken is dat we binnen een bepaalde doorlooptijd die opdrachten moeten afronden Informatie Belangrijke aspecten van informatie zijn: Vervuiling Op dit moment is het heel vervuild, het is belangrijk dat deze informatie up to date is, zo schoon mogelijk Rijkheid Meer informatie dat weergegeven zou worden om de mogelijke middelen te zien. Bijv. Koppeling aan een bekeuring systeem. Zodat we kunnen zien dat een persoon geen auto heeft, maar heel vaak in een auto van zijn neef rijdt en daarin verkeersovertredingen heeft gemaakt. Intelligentie Een persoon woont er niet, zaak wordt terug gekoppeld naar cjib. Cjib stuurt niet opdracht. Zou handig zijn als het systeem meedenkt met het koppelen van die persoon aan dezelfde agent die hem afgehandeld heeft. Afbreukrisico We willen de afbreukrisico van het uitvoeren van opdrachten zoveel mogelijk, door meer informatie en mogelijkheden. Je wilt alternatieven aanbieden om de afbreukrisico zoveel mogelijk te verminderen. Customizebility Je wilt de agent niet overspoelen met informatie. Mogelijkheid om weergave informatie t.b.v. de mogelijke alternatieven zelf te kunnen bepalen. Door middel van vinkjes o.i.d. Veiligheid Bij executie niet echt een kwestie Resources Je wilt het maximale uit je resources halen. Als iemand op bezoek gaat en niet alle opties uit heeft geprobeerd, dan is dat zonde want je moet die gozer dan weer op pad sturen. 120 Measuring the impact of information technology systems on police practices Politie agent Rotterdam Administratief werk Wat ik merk is dat we steeds vaker en steeds langer binnen moeten zitten om al het administratief werk te doen. We moeten ook teveel invoeren. Of we nou zon mobieltje krijgen. Met zon blackberry en zulke vingers kan ik echt niet al die gegevens invoeren. Dan moet ik toch thuis of op bureau gaan tikken. IT vs operationeel Hoe ik het zie is dat it gewoon komt, we krijgen het. En zij zien het vanuit een hun kortzichtige blik van dat dingen volgens hun alles er beter door zal gaan. Belang van informatie Informatie is te belangrijk. We hebben het nodig. Maar informatie moet compacter, in een oogopslag te zien. Inloggen is ook een probleem. We moeten dan inloggen op zon blackberry en ook voor de verschillende systemen een ander wachtwoord. Voor je het weet is die boef al lang weg hoor. Invoer vs uitvoer Wat tegenstrijdig is, is dat we niet willen dat we steeds meer in moeten voeren, want eig. betekent dit dat we minder op straat zijn. Maar anderzijds willen we wel heel graag de informatie als bron voor ons werk op straat. Veiligheid Volgens mij kan veiligheid alleen maar erop achteruit. Je staat op je mobiel te staren en je krijgt zo een klap voor je kop. Of ze trekken het uit je handen. Privacy/data risico Wat ik ook zie is dat we mobieltjes hebben en je laat het in de auto liggen. Als iemand zon mobiel te pakken krijgt, heeft ie zo goed als alle informatie van de politie toegang. Controle vs vrijheid In ons werk zijn wij nou eenmaal verbonden aan processen en controle van bovenaf dat dingen volgens een bepaalde manier moet. Of het nou meer of minder wordt, dat is nou eenmaal zo. Denhaag it ondersteuner Algemeen probleem In het huidig system hebben we te maken met mulderwet zaken, dus verkeersboetes. Maar ook met strafzaken, dus dat mensen hun straf uit moeten zitten. Wat nu veel voorkomt is dat als iemand gesignaleerd staat in dh, maar met zijn auto in Amsterdam rijdt. Als de politie hem checkt dan komt er niks te voorschijn (papos signalering) Information overload Wat interessant is, is dat zij hier in de buurt een tijd alle signalering vanuit papos naar ops hebben gezet. Dat zorgde dus voor info overload. De kenteken checkers hadden alsmaar hits. We kregen teveel hits, en hadden hier niet genoeg mankracht voor dus hebben we die informatie weer terug geschaald. Nadelen (recentheid van informatie) Wat nu gebeurt is dat informatie heen en weer gaat van cjib naar papos en ops enz. Dit duurt heel lang We hebben doorgaans ook te maken met lange doorlooptijden. De doorlooptijd van 4 weken is hier niet haalbaar. Hier heft executie prioriteit schaal 5, dus eig. wordt het alsmaar van tafel verschoven. Als je die lange doorlooptijd hebt met ook nog eens de verouderde informatie van de taken dan is het al helemaal lastig. Hier in zuiderpark hebben wij 8 wijken. 120 man beschikbaar. Eigenlijk zouden we de agenten moeten verplichten om de executielijsten na te kijken maar dat doen we niet. In de praktijk is het dan toch aan de deur gaan en als er iets niet klopt even naar centraal bellen. Beperktheid/ vrijheid Ik ben eig beperkt in het coördineren en dingen doen zodat alles efficiënter en sneller en beter kan door autorisatie issues. Ik mag het gewoon niet Duidend op: van bovenaf moet het perse op papier (executietaken lijst). Ik heb geen toegang tot Measuring the impact of information technology systems on police practices 121 bepaalde info. Ik mag niet met mijn excel ding aan de gang. Dus voor mij weer dubbel werk om deels in mijn excel ad slag te gaan maar ook weer terug naar papos. Voor de agent: Als agent wil je soms ook de vrijheid hebben om te kunnen handelen naar eigen inzicht. Als agent wil je soms barmhartig zijn en flexibel kunnen handelen. VS standaardisatie Aan de andere kant moet e rook geen ruimte zijn voor eigen inbreng. Want door de verschillende aanpakken krijg je het volgende gevolg: Gevolg: Dienstverlening naar de burger Als agent moet je gezag uitstralen. Als je inconsistent bent en onduidelijk ( de een zegt het 1 en de ander weer iets anders) dan gaat het ten koste van je uitstraling naar de burger. Er moet dus gewoon 1 manier zijn van handelen, dan weet de burger precies waar die aan toe is, je communiceert duidelijk naar de burger toe en je uitstraling en credibiliteit is daardoor behouden. Alternatieven voor dienstverlening naar de burger (aansluiting overheid) Dat je als politie in de zin van service verlenen meer services kan aanbieden. Bijv. Direct contact met maatschappelijk werker. I.p.v. Ga maar zoeken naar waar je een maatschappelijk werker kan vinden. Administratief werk (behoefte) Maak het makkelijk voor collega’s om in systemen te kunnen kijken. Het kost nu teveel moeite en tijd. Want minder administratief werk is meer blauw op straat. Beleving administratief werk ( wordt lastiger) We doen het nu op deze manier. Ook deels door de verschillende regels, maar ook misschien it. Het wordt niet eenvoudiger, het wordt alleen steeds meer en lastiger. Wijzend op al de codes enz. Beleving IT mensen vs operationeel Hier een service module, hier heb je het en gebruik het maar. Facilitaire ondersteuning. We krijgen een handleiding erbij gegooid. Laatst klopte deze handleiding ook niet eens. Zo nu en dan krijgen we een training, dit kan bijv. Veel vaker. De mensen bij IT luisteren niet naar wat wij te zeggen hebben. Ze vragen niet wat je nodig hebt. Verplichtigheid vs vrijwilligheid Nu voelt alles als verplicht aan en is er weinig vrijwilligheid in het invoeren van info. Besef invoeren/ foutief invoeren De huidige IT maakt lui. Agenten snappen niet hoe het proces in elkaar zit en vullen zomaar iets in. Je doet maar wat en volgt gewoon de velden die er staan. Toelichting Ruud: Nu kiest een agent maar een ander formulier i.p.v. het correcte formulier, omdat die gewoon minder velden heft om in te vullen. Controle Ingaand op het foutief invoeren van dingen ben ik er nog om te zorgen dat inderdaad de goede velden in zijn gevuld. Belang van info De huidige IT zorgt ervoor dat je info invult omdat het moet en niet omdat het belangrijk is om info in te voeren omdat het jou en anderen later helpt. IT moet triggeren om info uit te willen wisselen. Mensen moeten beseffen dat it er nou eenmaal is en dat het belangrijk is. Leren IT zou ook best kunnen helpen bij het opdoen van kennis. Spelenderwijs leren omgaan met it. Opbouwend, net als bij spelletjes. Kennis/ complexiteit van info Het kost ook zoveel moeite en met ib, meeste collega’s kennen de codes niet. Ik kan het ze ook uitleggen, maar de info blijft ook niet hangen. Als ik het doe dan is het binnen een paar minute gedaan. Als ik het uitleg dan duurt het tot een paar uur. Daarna vergeten ze het toch weer. Eigenlijk zou ik heel principieel moeten zijn en het de mensen ook gewoon uit te leggen om die mensen te dwingen het te begrijpen. Want dan weten ze dat ze het zelf ook uiteindelijk moeten doen. Drempel om IT te willen gebruiken ( acceptance ) Drastische verandering is pas goed genoeg om de agent te overtuigen van het belang en het willen werken met it Aansturing/productiviteit Ik ben degene die de agenten aanspreekt. Hoe kan het dat collega a al zoveel executieopdrachten hebben uitgevoerd en dat jij maar zoveel heb gedaan. Beetje concurrentie aspect benadrukken. Dat 122 Measuring the impact of information technology systems on police practices doe ik nu door het te benadrukken en het beetje persoonlijk te maken. Measuring the impact of information technology systems on police practices 123 

organogram W NL

VK GROUP, opgericht in 1952, is een toonaangevend Belgisch

Name Affiliation Albert van Dijk Australian National University

Medewerker financiële administratie,

eco eco eco eco eco eco

Ernst ten Heuvelhof - TU Delft Medewerkers

NL - BusinessForensics

Technische Robotica

Heynen Systems b.v., Lelystad, 11.07.2014 PERSBERICHT

Samenvatting Summary