You can help by commenting or suggesting your edit directly into the transcript. We'll review any changes before posting them. All comments are completely anonymous. For any comments that need a reply, consider emailing docs@inductiveautomation.com.
Version:
Supplemental Videos
LESSON
Reading a CSV File into a PyDataset
Description
See the step-by-step creation of a reusable script function to import CSV data into a PyDataset.
Video recorded using: Ignition 8.1
Resources
Transcript
(open in window)[00:00] In this lesson, we'll demonstrate how to create a new Python function in Ignition to prompt for and read a CSV file into a PyDataset. This function will be an initial step in the larger process of reading CSV data into a database using scripting, as we saw demonstrated in a prior lesson. The ultimate destination for this function will be a reusable project scripting library. This allows for reuse of the code for other purposes. However, we'll use the script console as a development workspace in which to iteratively write and test our code. So let's invoke it by clicking Tools, Script Console. Let's start by defining the needed function interface. We'll call our function read_csv_to_pydataset. It will prompt for a filename internally so it needs no inputs, but all following lines must be tab indented. Let's rough out the general execution flow as comments and add that next. We will prompt for a CSV filename, read its data, transform that data into a dataset, convert the dataset into a PyDataset for easier iteration, and then return the data and do any cleanup. Okay, let's flesh out this outline with the actual code. To read the CSV data, we'll start by prompting for the intended CSV file name. Ignition has a system function which does just that.
[01:34] So we'll use it and specify a CSV file type. Then when we have that file path, we'll open it using a Python system function and read it into a file object. Let's back up just a bit. Python has an existing module which handles CSV actions, so let's import that upfront. Then we can read the file object into a reader object.
[02:09] Let's remind ourselves of what our CSV data looks like. Our data has an initial header line, so we need to grab this, but exclude it from our data reading. So we'll use another Python system function and read the headers, then move ahead to the next line. Now we'll set up an empty data list, and now we're ready to loop over all rows in the reader object, where we'll append each row of data. And when finished, we're going to create a dataset using data headers and another Ignition system function. This might actually be a good point to spot check what we've written so far. Below this function, let's add a function call to the code above it. And internally, let's print the dataset we've just created. When we execute this, we are prompted for the CSV filename. We'll select the products CSV file, and in the right pane we see we've just read a 2 by 5 dataset, which is just what we expected. Only a few small steps remain.
[03:14] Let's convert the created dataset into a PyDataset, which is simply a wrapper which makes looping and indexing on a dataset a bit easier to perform. For this, we'll use another Ignition system function. And lastly, let's close the file, and return the PyDataset we've created. Our script console code looks complete now. So we can remove the internal print statement. Let's relocate the function to a scripting library. We will cut all lines except the last one, and create a new project script called toolbox. Then we'll paste our new function and save it. Then to do a final test of our function, let's return to our Script Console, clear the outputs, and reset the workspace.
[04:10] Now we'll need to call the function with a leading toolbox scope, and let's print the PyDataset and its first element using indexing. When we now execute this function, we are once more prompted for the CSV filename, and we return a 2 by 5 PyDataset and its first element that corresponds to our raw CSV file. So to recap this lesson, we've shown the steps to creating a reusable project library script, which prompts for a CSV filename, reads the data into a dataset, and then returns it as a PyDataset for ease of use. This can now be used as a standalone utility function, but we're going to make use of it to create the next needed function.