Summary and Schedule

ATTENTION This is an experimental test of The Carpentries Workbench lesson infrastructure. It was automatically converted from the source lesson via the lesson transition script.

If anything seems off, please contact Zhian Kamvar zkamvar@carpentries.org

Good data organization is the foundation of any research project. Most researchers have data in spreadsheets, so it’s the place that many research projects start.

Typically we organize data in spreadsheets in ways that we as humans want to work with the data. However computers require data to be organized in particular ways. In order to use tools that make computation more efficient, such as programming languages like R or Python, we need to structure our data the way that computers need the data. Since this is where most research projects start, this is where we want to start too!

In this lesson, you will learn:

• Good data entry practices - formatting data tables in spreadsheets
• How to avoid common formatting mistakes
• Approaches for handling dates in spreadsheets
• Basic quality control and data manipulation in spreadsheets
• Exporting data from spreadsheets

In this lesson, however, you will not learn about data analysis with spreadsheets. Much of your time as a researcher will be spent in the initial ‘data wrangling’ stage, where you need to organize the data to perform a proper analysis later. It’s not the most fun, but it is necessary. In this lesson you will learn how to think about data organization and some practices for more effective data wrangling. With this approach you can better format current data and plan new data collection so less data wrangling is needed.

Getting Started

Data Carpentry’s teaching is hands-on, so participants are encouraged to use their own computers to ensure the proper setup of tools for an efficient workflow.
These lessons assume no prior knowledge of the skills or tools.

To get started, follow the directions in the “Setup” tab to download data to your computer and follow any installation instructions.

Prerequisites

This lesson requires a working copy of spreadsheet software, such as Microsoft Excel or LibreOffice or OpenOffice.org (see more details in “Setup”).
To most effectively use these materials, please make sure to install everything before working through this lesson.

For Instructors

If you are teaching this lesson in a workshop, please see the Instructor notes.

	Setup Instructions	Download files required for the lesson
Duration: 00h 00m	1. Introduction	What are basic principles for using spreadsheets for good data organization?
Duration: 00h 18m	2. Formatting Data Tables in Spreadsheets	What are some common challenges with formatting data in spreadsheets and how can we avoid them?
Duration: 00h 48m	3. Formatting Problems	What are some common challenges with formatting data in spreadsheets and how can we avoid them?
Duration: 01h 08m	4. Dates as Data	What are good approaches for handling dates in spreadsheets?
Duration: 01h 28m	5. Quality Assurance	How can we carry out basic quality assurance in spreadsheets?
Duration: 01h 53m	6. Exporting Data	How can we export data from spreadsheets in a way that is useful for downstream applications?
Duration: 02h 08m	Finish

The actual schedule may vary slightly depending on the topics and exercises chosen by the instructor.

Data

You need to download some files to follow this lesson:

1. Download the following three files:

• SAFI_clean.csv
• SAFI_messy.xlsx
• SAFI_dates.xlsx

2. Place these 3 files in a folder you can easily find and access on your computer (for instance in a datacarpentry-spreadsheets folder on your Desktop or within your Home folder).

About the data

For more information about the dataset and to download it from Figshare, check out the Social Sciences workshop data page.

Software

To interact with spreadsheets, we can use LibreOffice, Microsoft Excel, Gnumeric, Onlyoffice, WPS office or other programs. Commands may differ a bit between programs, but the general ideas for thinking about spreadsheets are the same.

For this lesson, if you don’t have a spreadsheet program already, you can use LibreOffice. It’s a free, open source spreadsheet program.

macOS users who use Apple’s Numbers application should note that it does not contain some of the features (particularly data validation) that we will be using. Please use LibreOffice or Microsoft Excel instead.

Windows

• Download the Installer
• Install LibreOffice by going to the installation page. The version for Windows should automatically be selected. Click Download Version X.X.X (whichever is the most recent version).
• Install LibreOffice
• Once the installer is downloaded, double click on it and LibreOffice should install.

macOS

• Download the Installer
• Install LibreOffice by going to the installation page. The version for Mac should automatically be selected. Click Download Version X.X.X (whichever is the most recent version).
• Install LibreOffice
• Once the installer is downloaded, double click on it and LibreOffice should install.

Linux

• Download the Installer
• Install LibreOffice by going to the installation page. The version for Linux should automatically be selected. Click Download Version X.X.X (whichever is the most recent version).
• Install LibreOffice
• Once the installer is downloaded, double click on it and LibreOffice should install.
• package manager option:
  • pacman (Arch): pacman -S libreoffice
  • yum (Fedora, CentOS): yum install libreoffice
  • apt (Debian, Ubuntu): apt install libreoffice