Skip to main content

Cloud Database? NoSQL? Nah, just use CSVs and CsvHelper

KISS - Keep it Simple, Stupid. While I don't like calling people stupid, I do like to Keep it Super Simple!

I was talking to Jeff Fritz on my team about a new system we're architecting. I suggested CosmosDB or perhaps Azure Table Storage. Then we considered the amount of data we were storing (less than 100 megs) and Jeff said...let's just use CSV files and CsvHelper.

First I was shocked. SHOCKED I SAY.

via GIPHY

Then I was offended

via GIPHY

But finally I was hey...that's a good idea.

via GIPHY

A fine idea in fact. Why use more moving parts than needed? Sure we could use XML or JSON, but for our project we decided rather than even bother with an admin site that we'd use Excel for administration! It edits CSV files nicely thank you very much.

Can you parse CSV files yourself? Sure, but it'll start getting complex as you move between data types, think about quotes, deal with headers, whitespace, encoding, dates, etc. CSV files can be MUCH more complex and subtle than you'd think. Really.

Here's what CsvHelper can do for you:

var csv = new CsvReader( textReader );

var records = csv.GetRecords<MyClass>();

Here you just get an array of some class - if your class's structure maps 1:1 with your CSV file. If not, you can map your class with a projection of the types in the CSV file.

public sealed class PersonMap : CsvClassMap<Person>

{

    public PersonMap()

    {

        Map( m => m.Id );

        Map( m => m.Name );

        References<AddressMap>( m => m.Address );

    }

}


public sealed class AddressMap : CsvClassMap<Address>

{

    public AddressMap()

    {

        Map( m => m.Street );

        Map( m => m.City );

        Map( m => m.State );

        Map( m => m.Zip );

    }

}

And finally, just want to export a CSV from an Enumerable that mirrors what you want? Boom.

var csv = new CsvWriter( textWriter );

csv.WriteRecords( records );

Or do it manually if you like (hardcode some, pull from multiple sources, whatever):

var csv = new CsvWriter( textWriter );

foreach( var item in list )

{

    csv.WriteField( "a" );

    csv.WriteField( 2 );

    csv.WriteField( true );

    csv.NextRecord();

}

It won't replace SQL Server but it may just replace one-table SQLite's and "using a JSON file as a database" for some of your smaller projects. Check out CsvHelper's site and excellent docs here along with the CsvHelper GitHub here.

Sponsor: Check out JetBrains Rider: a new cross-platform .NET IDE. Edit, refactor, test and debug ASP.NET, .NET Framework, .NET Core, Xamarin or Unity applications. Learn more and download a 30-day trial!


© 2017 Scott Hanselman. All rights reserved.
     


from Scott Hanselman's Blog https://feeds.hanselman.com/~/456706218/0/scotthanselman~Cloud-Database-NoSQL-Nah-just-use-CSVs-and-CsvHelper.aspx
Labels:

Comments

Post a Comment

Popular posts from this blog

dotnet sdk list and dotnet sdk latest

Can someone make .NET Core better with a simple global command? Fanie Reynders did and he did it in a simple and elegant way. I'm envious, in fact, because I spec'ed this exact thing out in a meeting a few months ago but I could have just done it like he did and I would have used fewer keystrokes! Last year when .NET Core was just getting started, there was a "DNVM" helper command that you could use to simplify dealing with multiple versions of the .NET SDK on one machine. Later, rather than 'switching global SDK versions,' switching was simplified to be handled on a folder by folder basis. That meant that if you had a project in a folder with no global.json that pinned the SDK version, your project would use the latest installed version. If you liked, you could create a global.json file and pin your project's folder to a specific version. Great, but I would constantly have to google to remember the format for the global.json file, and I'd constan...
Post a Comment
Read more

Rail Fence Cipher Program in C and C++[Encryption & Decryption]

Here you will get rail fence cipher program in C and C++ for encryption and decryption. It is a kind of transposition cipher which is also known as zigzag cipher. Below is an example. Here Key = 3. For encryption we write the message diagonally in zigzag form in a matrix having total rows = key and total columns = message length. Then read the matrix row wise horizontally to get encrypted message. Rail Fence Cipher Program in C #include<stdio.h> #include<string.h> void encryptMsg(char msg[], int key){ int msgLen = strlen(msg), i, j, k = -1, row = 0, col = 0; char railMatrix[key][msgLen]; for(i = 0; i < key; ++i) for(j = 0; j < msgLen; ++j) railMatrix[i][j] = '\n'; for(i = 0; i < msgLen; ++i){ railMatrix[row][col++] = msg[i]; if(row == 0 || row == key-1) k= k * (-1); row = row + k; } printf("\nEncrypted Message: "); for(i = 0; i < key; ++i) f...
3 comments
Read more

Data Encryption Standard (DES) Algorithm

Data Encryption Standard is a symmetric-key algorithm for the encrypting the data. It comes under block cipher algorithm which follows Feistel structure. Here is the block diagram of Data Encryption Standard. Fig1: DES Algorithm Block Diagram [Image Source: Cryptography and Network Security Principles and Practices 4 th Ed by William Stallings] Explanation for above diagram: Each character of plain text converted into binary format. Every time we take 64 bits from that and give as input to DES algorithm, then it processed through 16 rounds and then converted to cipher text. Initial Permutation: 64 bit plain text goes under initial permutation and then given to round 1. Since initial permutation step receiving 64 bits, it contains an 1×64 matrix which contains numbers from 1 to 64 but in shuffled order. After that, we arrange our original 64 bit text in the order mentioned in that matrix. [You can see the matrix in below code] After initial permutation, 64 bit text passed throug...
Post a Comment
Read more