Skip to main content

5 posts tagged with "linux"

View All Tags

· 2 min read

I usually create a requirements.txt when I start a new Python project. I will do something like this:

# Create a new project
mkdir new-project
cd new-project

# Create requirements.txt
tee -a requirements.txt <<EOF
pandas
pyarrow
EOF

# Create a virtual environment and install dependencies
python -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip wheel setuptools
python -m pip install -r requirements.txt

This lets me get a new project up and running quickly. I have documented which packages I am using, which is good, but I have not documented which version of the package I am using. Since it is a new project, I want to use the latest version of every package. I could search PyPI for each package and find the latest version, but that takes a lot of time. Instead, I only document the package name in my initial requirements.txt. When I am ready to pin the versions, I use this bash script to check the version I have installed for each dependency listed in requirements.txt:

python -m pip freeze | grep -E $(cat requirements.txt | sed ':a;N;$!ba;s/\n/==|/g')

The result will be something like this:

pandas==2.1.4
pyarrow==14.0.2

· 2 min read

Reticulate is an R library that lets you execute Python code from within R. Recently, I attempted to use reticulate to access numpy from Python. Numpy was installed, but I kept getting the following message:

> reticulate::py_config()
python: /usr/home/sam.edwardes/rstudio-demos/applications/shiny-for-r-with-reticulate/.venv/bin/python3
libpython: /opt/python/3.10.11/lib/libpython3.10.so
pythonhome: /usr/home/sam.edwardes/rstudio-demos/applications/shiny-for-r-with-reticulate/.venv:/usr/home/sam.edwardes/rstudio-demos/applications/shiny-for-r-with-reticulate/.venv
version: 3.10.11 (main, Jun 4 2023, 22:34:21) [GCC 11.3.0]
numpy: [NOT FOUND]

· One min read

I always forget these commands... so here is my list of helpful commands to get information about your server.

  • Get your external IP address.
curl ifconfig.io 
  • Get your internal IP address. For example, this can be helpful if you are working with several EC2 instances that need to communicate with each other. You can use this instead of the external IP address for node to node communication. This has the benefit of being static even if you stop and start your EC2 instance.
hostname -i
  • Get information about your operating system.
lsb_release -a
  • Get information about your current operating system when in a Docker image.
cat  /etc/os-release 

· One min read

I often want to create new users in Linux for testing. Creating new users without interaction can be challenging to automate because the passwd command provides no way for you to pass in a plain text password. It will prompt you for a password which is OK for interactive sessions but not suitable for automation (e.g. creating new users in Pulumi).

NEW_USER_NAME=sam
useradd --create-home --home-dir /home/$NEW_USER_NAME -s /bin/bash $NEW_USER_NAME
passwd $NEW_USER_NAME
# New password:
# Retype new password:
# passwd: password updated successfully

The solution I have found is to pipe the password into the passwd command like this:

NEW_USER_NAME=sam
NEW_USER_PASSWORD=password
useradd --create-home --home-dir /home/$NEW_USER_NAME -s /bin/bash $NEW_USER_NAME
echo -e "${NEW_USER_PASSWORD}\n${NEW_USER_PASSWORD}" | passwd $NEW_USER_NAME

This trick allows you to create new users and set their passwords without interaction!