McDonald, John, and Valerie Lévillé. "Wither the Retention Schedule in the Era of Big Data and Open Data?" [Forthcoming in Records Management Journal).
Existing Citations
big data : The concept of big data goes beyond volume. Big data can be defined as large quantities and varieties of data that, due to their fast and sometimes ‘real-time’ availability, require extensive manipulation and mining through the intervention of various non-traditional technologies and tools. When properly mined, combined, manipulated, and analyzed, this information can lead to new and better analytics, easier and more opportunities for validation and faster insights (Vriens, 2013). The concept of big data can be summarized by "three Vs": volume, variety and velocity. (†816)
big data : Unlike open data initiatives where the objective is to make data produced using public funds available to a wide variety of audiences, the objective of big data initiatives is to help organizations exploit the value of the often voluminous and repetitive data that may reside in their databases to pursue their strategic and operational priorities and goals. These could include achieving a competitive advantage, enhancing services, improving business productivity, generating new economic value, maximizing marketing opportunities, improving outreach, supporting development and planning, or holding the organization to account. (†817)
big data : Big data initiatives differ from open data initiatives in a number of important ways, beginning with their objectives. The objectives of big data initiatives generally focus on the value of data to the interests of an individual organization or a collection of organizations (e.g. a partnership) in terms of its operational and/or strategic priorities. Open data initiatives on the other hand focus on the value of data to external interests in response to public policies on openness and transparency, coupled with the interests of various industry sectors in using and reusing publically funded data for economic and social development. Big data initiatives often involve the development of new processes and systems designed to extract, combine, manipulate and otherwise exploit data from existing systems, while open data initiatives tend to be based on existing datasets, small databases, and statistics that are packaged for dissemination or access through a portal. Furthermore, big data initiatives are seen in both private and public sector organizations, while open data initiatives tend to be supported by the public sector. (†818)
big data : The extent to which the data used to support big data initiatives can be trusted is dependent upon the quality, integrity and completeness of the controls that are in place to manage the data that is contributing to these initiatives. These controls are normally established as a result of the steps involved in planning, designing, testing, implementing, maintaining and reviewing the systems responsible for generating and managing the data. Although data security, protection of privacy, and the availability of storage space are significant challenges for big data initiatives, the biggest and perhaps the root challenge lies in building standards and processes that enable the large volumes of data to be manipulated in a manner that will ensure that the results, the final output will have integrity. (†821)
open data : The concept of open data is twofold; it couples the act of proactive disclosure of government-generated information in the form of open datasets, normally within the context of Open Government policy, and the intended audience’s ability to access that data. The concept is rooted in the objective to increase government transparency, generate public input and interest, and stimulate social and economic development. (†813)
open data : Big data initiatives differ from open data initiatives in a number of important ways, beginning with their objectives. The objectives of big data initiatives generally focus on the value of data to the interests of an individual organization or a collection of organizations (e.g. a partnership) in terms of its operational and/or strategic priorities. Open data initiatives on the other hand focus on the value of data to external interests in response to public policies on openness and transparency, coupled with the interests of various industry sectors in using and reusing publically funded data for economic and social development. Big data initiatives often involve the development of new processes and systems designed to extract, combine, manipulate and otherwise exploit data from existing systems, while open data initiatives tend to be based on existing datasets, small databases, and statistics that are packaged for dissemination or access through a portal. Furthermore, big data initiatives are seen in both private and public sector organizations, while open data initiatives tend to be supported by the public sector. (†819)
open data : The reliability, accuracy and trustworthiness of the datasets used in support of open data initiatives depends on the ability to trace them back to the original information sources from which they have derived. The ability to demonstrate the trustworthiness of the original record sources – that is, being able to show that the record sources are complete, authentic, and managed within a secure and controlled environment – in turn helps promote confidence that the data sets generated from these sources can be trusted. Conversely, the absence of controls that would otherwise support and ensure the trustworthiness of the original source records will erode trust in the derived data sets and, as a consequence, undermine the quality and effectiveness of the initiatives that are using the datasets. (†820)
retention : While the objectives, communities and audiences of big data and open data initiatives may vary to some degree, they share one common and very important characteristic that is fundamental to understanding how the issues of retention and disposition should be approached: they are based on some form of business or work process. (†822)
retention schedule : It is one thing to base the development of retention and disposition specifications on the business process, but quite another to decide which records should be retained and for how long. The study emphasized the need for criteria that would help in deciding which records generated by which transactions in a given business process should be retained and for how long. Based on the guidance provided in the ISO technical report, the study suggested that rather than focus on the records, the focus should be on the transactions that collectively comprise a given business process, understanding that transactions include computer transactions and administrative transactions (i.e.: actions between two or more parties). (†824)
retention schedule : The schedule, a formal document describing the records, their retention periods (the active, semi-active and inactive periods of their life cycle), and the manner of their disposal (destruction or transfer to long term preservation), serves to provide the following benefits: “increase control and standardization; ensure rapid access and retrievability; enhance management decision-making capabilities; foster a corporate compliance culture; demonstrate corporate accountability; decrease vicarious accountability and streamline redundancy and optimize business processes” (Myler, 2006). Although listings of the records could serve as a basis for recording the retention periods, the organization’s file classification system was often used as the basis of the schedule. (†823)
