Search code examples
javascriptarrayscsvsortingqualtrics

Load and process csv file in javascript within Qualtrics


I am trying to load a csv file within Qualtrics's javascript code and filter it based on the values within it. My csv file was created with the following r script:

d <- structure(list(U = c("a", "b", "c", "d", "e", "f"), 
                    T = c(1, 2, 3, 4, 5, 6), 
                    aE = c("X", "Y", "Y", "Y", "Y", "Y"), 
                    aA = c("R", "S", "S", "T", "U", "V"), 
                    aX = c("P", "Q", "Q", "W", "P", "P"), 
                    aP = c("Z", "A", "A", "A", "Z", "A"), 
                    aJ = c("K", "L", "L", "K", "K", "L"), 
                    aD = c("C", "B", "B", "B", "D", "D"), 
                    bE = c("X", "Y", "Y", "X", "X", "Y"), 
                    bA = c("G", "R", "R", "I", "I", "T"), 
                    bX = c("M", "N", "N", "O", "M", "O"), 
                    bP = c("Z", "A", "A", "Z", "A", "Z"), 
                    bJ = c("K", "L", "L", "L", "K", "L"), 
                    bD = c("B", "C", "C", "C", "B", "C")), 
               row.names = c(NA, -6L), 
               class = c("tbl_df", "tbl", "data.frame"))

write.csv(d, "data.csv")

The csv file looks like the following:

image1

EDIT

The csv file in a text editor is the following:

"","U","T","aE","aA","aX","aP","aJ","aD","bE","bA","bX","bP","bJ","bD"
"1","a",1,"X","R","P","Z","K","C","X","G","M","Z","K","B"
"2","b",2,"Y","S","Q","A","L","B","Y","R","N","A","L","C"
"3","c",3,"Y","S","Q","A","L","B","Y","R","N","A","L","C"
"4","d",4,"Y","T","W","A","K","B","X","I","O","Z","L","C"
"5","e",5,"Y","U","P","Z","K","D","X","I","M","A","K","B"
"6","f",6,"Y","V","P","A","L","D","Y","T","O","Z","L","C"

I then load it in javascript and attempt to process it using the following code:

$(document).ready(function() {
    var task = '1';
    var userID = 'a';

    fetch('https://raw.githubusercontent.com/username/repo/main/data.csv')
        .then(response => response.text())
        .then(csvData => {
            // Parse CSV data into an array of rows
            var rows = csvData.trim().split('\n');

            // Extract headers and trim whitespace
            var headers = rows[0].split(',').map(header => header.trim());

            // Extract and parse each row (excluding headers)
            var data = rows.slice(1).map(row => {
                var rowValues = row.split(',').map(value => value.trim());
                var rowData = {};

                headers.forEach((header, index) => {
                    rowData[header] = rowValues[index];
                });

                return rowData;
            });

            console.log('Parsed Data:', data);

            // Filter data to only include rows that match the specified task and userID
            var filteredData = data.filter(row => row['T'] === task && row['U'] === userID);

            console.log('Filtered Data:', filteredData);

            // Display the filtered row in the HTML document
            if (filteredData.length > 0) {
                var resultRow = filteredData[0]; // Get the first (and only) matching row
                displayResultRow(resultRow); // Call function to display the row in the document
            } else {
                console.log('No matching row found.');
                // Handle case where no matching row is found
            }

            return filteredData; // Return the filtered array of data objects
        })
        .then(filteredData => {
            console.log('Returned Filtered Data:', filteredData);
            // Use the returned filtered array of data objects here or perform additional actions
        })
        .catch(error => {
            console.error('Error fetching or processing CSV:', error);
            // Handle fetch or processing errors
        });

    function displayResultRow(row) {
        // Assuming there is a container in your HTML document with id="resultContainer"
        var resultContainer = document.getElementById('resultContainer');
        
        // Clear previous content
        resultContainer.innerHTML = '';

        // Create and append elements to display the row data
        Object.keys(row).forEach(key => {
            var rowElement = document.createElement('div');
            rowElement.textContent = `${key}: ${row[key]}`;
            resultContainer.appendChild(rowElement);
        });
    }
});

The initial data loading works, as I am able to successfully see the data object. However none of the processing works successfully. I believe this is because the columns and rows may not be created correctly. This is how the data object looks in my js code editor:

Parsed Data:
(6) [{...}, {...}, {...}, {...}, {...}, ...]
0
:
(15) {"": ""1"", "U": ""1...}
1
:
(15) {"": ""2"", "U": ""2...}
2
:
(15) {"": ""3"", "U": ""3...}
3
:
(15) {"": ""4"", "U": ""4...}
4
:
(15) {"": ""5"", "U": ""5...}
5
:
(15) {"": ""6"", "U": ""6...}

This seems to me like the row numbers are being read in, and the column headers are treated as other values of some sort. In any case, the data filtering is not working. I also am unsure how to store the data object to access it.

How can I take the csv file that I successfully load into javascript, process it to select the only row that has U = the specified userID above, and T = the specified task above? How can I store this row as an object (say, entitled info_object) so that I can then extract the specific values of particular columns within that row (ex: save the value of aE as the object "info_aE")?


Solution

  • Here is a working snippet demonstrating the use of yet another CSV parser library:

    const text=`"","U","T","aE","aA","aX","aP","aJ","aD","bE","bA","bX","bP","bJ","bD"
    "1","a",1,"X","R","P","Z","K","C","X","G","M","Z","K","B"
    "2","b",2,"Y","S","Q","A","L","B","Y","R","N","A","L","C"
    "3","c",3,"Y","S","Q","A","L","B","Y","R","N","A","L","C"
    "4","d",4,"Y","T","W","A","K","B","X","I","O","Z","L","C"
    "5","e",5,"Y","U","P","Z","K","D","X","I","M","A","K","B"
    "6","f",6,"Y","V","P","A","L","D","Y","T","O","Z","L","C"`;
    
    // in the actual situation you would retrieve the text 
    // from your file, using asynchronous fetch(URL).then(r=>r.text())
    // In a further .then(text=>{...}) you would place the following:
    const res=Papa.parse(text,{header:true,dynamicTyping:true}).data;
     console.log(JSON.stringify(res));
    <script src="https://cdnjs.cloudflare.com/ajax/libs/PapaParse/5.4.1/papaparse.min.js"></script>

    Alternatively you could retrieve the text directly from your URL with

    Papa.parse(URL,{download:true,header:true,dynamicTyping:true,complete:({data:res})=>{
     console.log(JSON.stringify(res));
    }
    });