Seed variety purity is the main indicator of seed quality, which affects crop yield and product quality. In the present study, a new method for the identification of pine nut varieties based on hyperspectral imaging and convolutional neural networks LeNet-5 was established so as to avoid the hybridization of different varieties of pine nuts, improve the identification efficiency and reduce the cost of identification. Images of 128 wavelengths in the 370–1042 nm range were acquired by hyperspectral imaging. The spectrum and image of each seed were obtained by means of black-and-white correction and region segmentation of the original image. Twenty characteristic wavelengths were extracted from the first three principal components (PCs) of principal component analysis (PCA). A support vector machine (SVM) spectral recognition model based on full wavelengths and characteristic wavelengths was established. For different species of pine seeds, the classification accuracies of the prediction set in the aforementioned datasets were 97.7 and 93.1%, respectively. The seed images of 20 characteristic wavelengths were input into LeNet-5 to improve the network structure and the number of convolution channels. The improved LeNet-5 performed better with over 99% accuracy. Such results show that the convolutional neural network is of considerable significance for fast and nondestructive identification of pine seed varieties.