The inside-out sequential procedures for testing up to k upper outliers in a two-parameter exponential sample are investigated. Six test statistics, one based on the ratio of the difference of largest observation and the sample mean which are unsuspected to be outliers to the range of these observations, and others used for block test procedures discussed in Basu (J Am Stat Assoc 60:548–559, 1965), Balasooriya and Gadag (J Stat Comput Simul 50:249–259, 1994), Zerbet and Nikulin (Commun Stat Theory Methods 32:573–583, 2003) and Kumar (Testing for suspected observations in an exponential sample with unknown origin, 2013), are considered. Utilizing the recursion of Huffer (J Appl Probab 25:346–354, 1988) and algorithm of Lin and Balakrishnan (Comput Stat Data Anal 53:3281–3290, 2009), the critical values of the joint null distributions of these test statistics for sequential testing discordancy of k upper outliers in two-parameter exponential samples on the important cases k=2 and 3 are obtained. We also propose a simple procedure to determine k, which can reduce the masking or swamping effect. Powers of tests based on these statistics are compared through a Monte Carlo study.