Try something like this:

function xxx() {
    // Get table element
    let table = document.getElementById("mytable");
    // Get table rows
    let rows = table.rows;
    let bool = false;

    // Go over rows
    for(i = 0; i < rows.length; i++) {
        // Get cells of current row
        const columns = rows[i].cells;
        let count = 0;

        // Go over current row's cells
        for(j = 0; j < columns.length; j++) {
            // Count a cell if it has a value
            if(columns[j].children[0].value != "") {
                count++;
            }
        }
        bool = count <= 5;
    }
    console.log(bool);
}

The function prints true if all rows have no more than 5 values per line.

I wrote an html table filled with inputs. How could I use JavaScript to check if no more than 5 numbers were entered per line.

This is an example row

 <table id="mytable" onchange="xxx()">   
<tr>
    <td><input type="number" name="num1-1" min="1" max="10" ></td>
    <td><input type="number" name="num2-1" min="11" max="20" ></td>
    <td><input type="number" name="num3-1" min="21" max="30" ></td>
    <td><input type="number" name="num4-1" min="31" max="40" ></td>
    <td><input type="number" name="num5-1" min="41" max="50" ></td>
    <td><input type="number" name="num6-1" min="51" max="60" ></td>
    <td><input type="number" name="num7-1" min="61" max="70" ></td>
    <td><input type="number" name="num8-1" min="71" max="80" ></td>
    <td><input type="number" name="num9-1" min="81" max="90" ></td>
</tr>
</table>