Enabling UTF-8 support on the command line¶
When we're using the command line, we might assume that we can type just about anything. We might want to type things like:
- cupcakes
- crème brûlée
- アイスクリーム
- шоколад
- 🍩🍩🍩
That's freedom, right? But it turns out you probably can't type all of those things!
Your computer might be only set up for English text, or maybe it doesn't like accent marks or Japanese, or maybe 🍰 or 🍦 or 🍭 are just too much for it to handle!
Computers that only really understand English and Western European languages are set up with something called Latin-1, which is a very short list of characters that can be displayed. To make sure our command line can understand (and display) all text around the world – plus emoji – we're going to set up a super long list of characters UTF-8.
Setting up UTF-8 support on macOS¶
macOS by default supports UTF-8 in the command line. But… sometimes it doesn’t? So we’ll add a few commands that will run every time you start the shell that forces UTF-8.
For the commands below: if you’re typing instead of cutting and pasting, make sure you use
>>— two greater-than symbols — not». Also, be sure to hit enter after pasting in the second line.
To start, type echo $SHELL and press enter. If it says /bin/zsh run the following command:
But it says /bin/bash run the following commands:
echo 'export LC_ALL=en_US.UTF-8' >> ~/.bash_profile
echo 'export LANG=en_US.UTF-8' >> ~/.bash_profile
If you do it right, it won’t say anything. But I promise it worked!
Setting up UTF-8 support on Windows¶
Good news: if you're using Git Bash inside of Windows Terminal, the command line itself already understands UTF-8. You can test it by running this command:
You should see a bunch of nice-looking text in all sorts of languages and symbols.
Python is a different story. On Windows, Python normally reads and writes files using an old Windows-only encoding instead of UTF-8. That's how you end up with UnicodeDecodeError messages, or CSVs where é turns into é.
If you followed the Python setup instructions this is already fixed! We added export PYTHONUTF8=1 to your ~/.bashrc, which tells Python to use UTF-8 for everything. To double-check, run:
If it prints 1, you're all set. If it prints 0, run the following command, then close your command line and open a new one: