Find all the values of x that are not in the domain of f.

The values of x that are not in the domain of f are those that cannot be used because they leave an undefined output value, that means those that that leave the denominator of the function equal to zero.
[tex]x^2-3x-40=0[/tex]Factoring we have:
[tex](x+5)(x-8)=0[/tex]This expression is zero if any of the factors is zero, then:
[tex]x+5=0[/tex][tex]x=-5[/tex]Or
[tex]x-8=0[/tex][tex]x=8[/tex]The values of x that are not in the domain of f are -5 and 8.