Denote by T-n(d)(A) an upper triangular operator matrix of dimension n is an element of N whose diagonal entries D-i, 1 <= i <= n, are known, and A = (A(ij))(1 <= i<j <= n) is an unknown tuple of operators. This article is aimed at investigation of defect spectrum D-sigma* = boolean OR(n)(i=1) sigma*(D-i)\sigma(*)(T-n(d)(A)), where sigma(*) is a spectrum corresponding to various types of invertibility: (left, right) invertibility, (left, right) Fredholm invertibility, left/right Weyl invertibility. We give characterizations for each of the previous types, and provide some sufficent conditions for the stability of certain spectrum (the case D-sigma* = empty set). The results are proved for all matrix dimensions n >= 2, and they hold in arbitrary Hilbert spaces without assuming separability, thus generalizing results fromWu and Huang (Ann Funct Anal 11(3):780-798, 2020; Acta Math Sin 36(7):783-796, 2020). We also retrieve a result from Bai et al. (J Math Anal Appl 434(2):1065-1076, 2016) in the case n = 2, and we provide a precise form of the well known 'filling in holes' result from Han et al. (Proc Am Math Soc 128(1):119-123, 2000).