(1) L0 norm (\(\|\theta\|_0\))

The zero norm (also known as \(l_0\) norm, L0 norm) of a vector \(\theta \in \{1,...,d\}\) is denoted \(\|\theta\|_0\) and is defined as follows:

\begin{equation} \label{eq: l0} \lVert\theta\lVert_0 = \sum_{j=1}^d \mathbb{I}(\theta_j \neq 0) \end{equation}

where \(\mathbb{I}(\cdot)\) is an indicator function. Thus, L0 norm indicate the number of nonzero elements in \(\theta\).

(2) L1 norm (\(\|\theta\|_1\))

The one norm (also known as the \(l_1\) norm, L1 norm, or mean norm) of a vector \(\theta \in \{1,...,d\}\) is denoted \(\|\theta\|_1\) and is defined as follows:

\begin{equation} \label{eq: l1} \lVert\theta\lVert_1 = \sum_{j=1}^d \lvert\theta_j\lvert \end{equation}

Thus, L1 norm is defined as the sum of the absolute values of its elements.

(3) L2 norm (\(\|\theta\|_2\))

The two norm (also known as the \(l_2\) norm, L2 norm, square norm, or Euclidean norm) of a vector \(\theta \in \{1,...,d\}\) is denoted \(\|\theta\|_2\) and is defined as follows:

\begin{equation} \label{eq: l2} \lVert\theta\lVert_2 = \sqrt{\sum_{j=1}^d \lvert\theta_j\lvert^2} \end{equation}

L2 norm is defined as the square root of the sum of the squares of the absolute values of its elements.

(4) Infinity norm (\(\|\theta\|_{\infty}\))

The infinity norm (also known as the \(l_{\infty}\) norm, max norm, or uniform norm) of a vector \(\theta \in \{1,...,d\}\) is denoted \(\|\theta\|_2\) and is defined as follows:

\[\begin{equation} \label{eq: linf} \|\theta\|_{\infty} = \max_{j=1,...,d} \{|\theta_j|\} \end{equation}\]

\(L_{\infty}\) norm is defined as the maximum of the absolute values of its elements.

(5) Lp norm (\(\|\theta\|_{p}\))

The p norm (also known as the \(l_p\) norm, Lp norm) of a vector \(\theta \in \{1,...,d\}\) is denoted \(\|\theta\|_p\) and is defined as follows:

\begin{equation} \label{eq: lp} \lVert\theta\lVert_p = \sqrt[p]{\sum_{j=1}^d \lvert\theta_j\lvert^p} \end{equation}