Approximating a binomial distribution by a suitable normal distribution is a well known practice, and widely discussed in introductory level statistics books. Recently it has been shown (in Chang et al. (2008)) that the skew-normal distributions can provide a far better approximation than the normal ones. This article revisits the approximation issue and other related inferential aspects. Though we are restricting ourselves here to binomial distribution only, our investigation shows that the same patterns hold good for Poisson, negative binomial and hypergeometric distributions as well.