Edit

Manually create and use a Linux NFS (Network File System) Server with Azure Kubernetes Service (AKS)

  • Article

Sharing data between containers is often a necessary component of container-based services and applications. You usually have various pods that need access to the same information on an external persistent volume. While Azure Files is an option, creating an NFS Server on an Azure VM is another form of persistent shared storage.

This article will show you how to create an NFS Server on an Azure Ubuntu virtual machine, and set up your AKS cluster with access to this shared file system as a persistent volume.

Before you begin

This article assumes that you have the following to support this configuration:

If you deploy your AKS cluster first, Azure automatically populates the virtual network settings when deploying your Azure Ubuntu VM, associating the Ubuntu VM on the same VNet. If you want to work with peered networks instead, consult the documentation above.

Deploying the NFS Server onto a virtual machine

  1. To deploy an NFS Server on the Azure Ubuntu virtual machine, copy the following Bash script and save it to your local machine. Replace the value for the variable AKS_SUBNET with the correct one from your AKS cluster, otherwise the default value specified opens your NFS Server to all ports and connections. In this article, the file is named nfs-server-setup.sh.

    #!/bin/bash

# This script should be executed on Linux Ubuntu Virtual Machine

EXPORT_DIRECTORY=${1:-/export/data}
DATA_DIRECTORY=${2:-/data}
AKS_SUBNET=${3:-*}

echo "Updating packages"
apt-get -y update

echo "Installing NFS kernel server"

apt-get -y install nfs-kernel-server

echo "Making data directory ${DATA_DIRECTORY}"
mkdir -p ${DATA_DIRECTORY}

echo "Making new directory to be exported and linked to data directory: ${EXPORT_DIRECTORY}"
mkdir -p ${EXPORT_DIRECTORY}

echo "Mount binding ${DATA_DIRECTORY} to ${EXPORT_DIRECTORY}"
mount --bind ${DATA_DIRECTORY} ${EXPORT_DIRECTORY}

echo "Giving 777 permissions to ${EXPORT_DIRECTORY} directory"
chmod 777 ${EXPORT_DIRECTORY}

parentdir="$(dirname "$EXPORT_DIRECTORY")"
echo "Giving 777 permissions to parent: ${parentdir} directory"
chmod 777 $parentdir

echo "Appending bound directories into fstab"
echo "${DATA_DIRECTORY}    ${EXPORT_DIRECTORY}   none    bind  0  0" >> /etc/fstab

echo "Appending localhost and Kubernetes subnet address ${AKS_SUBNET} to exports configuration file"
echo "/export        ${AKS_SUBNET}(rw,async,insecure,fsid=0,crossmnt,no_subtree_check)" >> /etc/exports
echo "/export        localhost(rw,async,insecure,fsid=0,crossmnt,no_subtree_check)" >> /etc/exports

nohup service nfs-kernel-server restart

    The script initiates a restart of the NFS Server, and afterwards you can proceed with connecting to the NFS Server from your AKS cluster.

  2. After creating your Linux VM, copy the file created in the previous step from your local machine to the VM using the following command:

    scp /path/to/nfs-server-setup.sh username@vm-ip-address:/home/{username}

  3. After the file is copied over, open a secure shell (SSH) connection to the VM and execute the following command:

    sudo ./nfs-server-setup.sh

    If execution fails because of a permission denied error, set execution permission for all by running the following command:

    chmod +x ~/nfs-server-setup.sh

Connecting AKS cluster to NFS Server

You can connect to the NFS Server from your AKS cluster by provisioning a persistent volume and persistent volume claim that specifies how to access the volume. Connecting the two resources in the same or peered virtual networks is necessary. To learn how to set up the cluster in the same VNet, see: Creating AKS Cluster in existing VNet.

Once both resources are on the same virtual or peered VNet, provision a persistent volume and a persistent volume claim in your AKS Cluster. The containers can then mount the NFS drive to their local directory.

  1. Create a YAML manifest named pv-azurefilesnfs.yaml with a PersistentVolume. For example:

    apiVersion: v1
kind: PersistentVolume
metadata:
  name: NFS_NAME
  labels:
    type: nfs
spec:
  capacity:
    storage: 1Gi
  accessModes:
    - ReadWriteMany
  nfs:
    server: NFS_INTERNAL_IP
    path: NFS_EXPORT_FILE_PATH

    Replace the values for NFS_INTERNAL_IP, NFS_NAME and NFS_EXPORT_FILE_PATH with the actual settings from your NFS Server.

  2. Create a YAML manifest named pvc-azurefilesnfs.yaml with a PersistentVolumeClaim that uses the PersistentVolume. For example:

    Important

    storageClassName value needs to remain an empty string or the claim won't work.

    apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: NFS_NAME
spec:
  accessModes:
    - ReadWriteMany
  storageClassName: ""
  resources:
    requests:
      storage: 1Gi
  selector: 
    matchLabels:
      type: nfs

    Replace the value for NFS_NAME with the actual setting from your NFS Server.

Troubleshooting

If you can't connect to the server from your AKS cluster, the issue might be the exported directory or its parent, doesn't have sufficient permissions to access the NFS Server VM.

Check that both your export directory and its parent directory are granted 777 permissions.

You can check permissions by running the following command and the directories should have 'drwxrwxrwx' permissions:

ls -l

Next steps