Interactive oracle proofs (IOPs) (Ben-Sasson et al., in: Hirt and Smith (eds) TCC 2016-B, Part II, volume 9986 of LNCS, pp 31-60, 2016. https://doi.org/10.1007/978-3-662-53644-5_2; Reingold et al. in SIAM J Comput, 2021) have emerged as a powerful model for proof systems which generalizes both Interactive Proofs (IPs) and Probabilistically Checkable Proofs (PCPs). While IOPs are not any more powerful than PCPs from a complexity theory perspective, their potential to create succinct proofs and arguments has been demonstrated by many recent constructions achieving better parameters such as total proof length, alphabet size, and query complexity. In this work, we establish new results on the relationship between various notions of soundness for IOPs. First, we formally generalize the notion of round-by-round soundness (Canetti et al., in: Charikar and Cohen (eds) 51st ACM STOC, pp 1082-1090, 2019. https://doi.org/10.1145/3313276.3316380) and round-by-round knowledge soundness (Chiesa et al., in: Hofheinz and Rosen (eds) TCC 2019, Part II, volume 11892 of LNCS, pp 1-29, 2019. https://doi.org/10.1007/978-3-030-36033-7_1). Given this generalization, we then examine its relationship to the notions of generalized special soundness (Attema et al., in: Malkin and Peikert (eds) CRYPTO 2021, Part IV, volume 12828 of LNCS, pp 65-91, Virtual Event, August 2021, 2021. https://doi.org/10.1007/978-3-030-84259-8_3; Malkin and Peikert (eds) CRYPTO 2021, Part II, volume 12826 of LNCS, pp 549-579, Virtual Event, August 2021, 2021. https://doi.org/10.1007/978-3-030-84245-1_19) and generalized special unsoundness (Attema et al., in: Kiltz and Vaikuntanathan (eds) Theory of Cryptography - 20th International Conference, TCC 2022 Chicago, IL, USA, November 7-10, 2022, Proceedings, Part I, 2022). We show that: generalized special soundness implies generalized round-by-round soundness;generalized round-by-round knowledge soundness implies generalized special soundness;generalized special soundness does not imply generalized round-by-round knowledge soundness;generalized round-by-round soundness (resp., special unsoundness) is an upper bound (resp., a lower bound) on standard soundness, and this relationship is tight when the round-by-round soundness and special unsoundness errors are equal; andany special sound IOP can be transformed via (a variant of) the Fiat-Shamir transformation (in the Random Oracle Model) into a non-interactive proof that is adaptively sound in the Quantum Random Oracle Model.