Machine Learning Methods in Geosciences (in progress)

Instructors: Jerry Schuster

Books: Machine Learning Methods in Geosciences (in progress) by G.T. Schuster

Suggested Readings: Deep Learning by Goodfellow et al. (2016), tutorials, Neural Networks and Learning by Haykin (2009), and Learning from Data: A Short Course by Mostafa, Magdon-Ismail, and Lin (2012), Pattern Recognition and ML by Bishop (2006).

Objective: Learn fundamental concepts of machine learning and their applications in geosciences. The course concentrates on the theories and applications of neural networks, convolutional neural networks (CNN), support vector machines, principal component analysis, and cluster methods. Each method is accompanied by MATLAB exercises, most of which are applied to geophysical data.

Audience:The course is for geoscientists who have heard about ML and might know some details, but do not have a very deep or comprehensive knowledge about the theory or how it can be applied to oil exploration problems. A selective overview of ML topics is provided and the depth of understanding comes from the MATLAB exercises that allow the user to apply ML methods to geoscience problems such as semblance velocity analysis, multiple elimination, dispersion curve picking, salt picking, denoising of migration images, and feature detection in aerial photos. The last day of the course provides the theoretical connection between sparse inversion and neural networks, which might benefit the ML expert who has not yet been exposed to this recent development. It provides a rational understanding of some properties of CNN, such as why larger features start to appear in the deeper portions of a network and the relationship between the number of sparse inversion iterations and the depth of a neural network.

Format: Five days of lectures 4 hours/day. Lab exercises 3-4 hours/day.

MATLAB Labs: Computational Labs

Youtube Labs: YouTube videos are listed.

DAY

SUBJECT

READING

HMWK/LAB

Day 1

  • Course Overview,
  • Choosing ML Algorithm,
  • Quasi-elastic Wave Equation and AVO
  • CSIM Overview.

    • Chapters 1-2.

    Exercises: Exercises 2.1-2.7 and 2.9.

    YouTube: Machine Learning Overview, Linear Model I: Linear Regression, and Learning.

    Day 1

  • Intro Steepest Descent & NN
  • Activation Functions.

    • Chapters 2-3

    Exercises: Chapter 3: 3.1, 3.2, 3.3, 3.7, 3.8, and 3.9.

    YouTube: Linear Model I: Linear Regression, ML: Training vs Testing and Linear Model II.

    Day 2

  • Single Node Neural Network, Multiple Node Neural Network, Recursive Propagation, Soft-Max Multinomial Squasher, cross-entropy error function

    • Chapter 4

    Exercises: Exercises 4.1-4.7 and Fully Connected NN Matlab Lab as Multinary Classifier.

    Papers: Fukushima (1980), Hubel+Wiesel (1958), LeCun et al. (1998), 9 key papers, and Aramco (2018) papers.

    YouTube: Neural Networks.

    Day 2

  • Preconditioning NN and CNN Data,
  • Generalized Neural Networks and MATLAB.

    • Chapter 4-5

    Exercises: Fully Connected NN Matlab Lab as Binary Classifier. The PPT that explains the code is here.

    Day 2

  • CNN,
  • Accelerated SD,
  • CNN+SLIC Picking Objects in Drone Photos,
  • CNN Salt Picking& in Migration Image,
  • Deconvolution CNN

    • Chapter 5

    Exercise: Number Reading CNN Lab.

    YouTube: Q&D CNN

    Papers: ADAM Step Length, CNN Overview Papers, TLE Picking Salt Boundaries by CNN, CSIM SLIC Picker, and Shihang's SEG Picking Salt Boundary by CNN, Deep Deconvnet for Segmentation, DCNN Segmentation.

    Day 3

  • CNN Understanding,
  • Dual Lagrange Problem,
  • Salt Picking by Texture+Cluster,
  • Eduardo Traveltime Picker, and
  • Kai Traveltime Picker.

    • Chapters 6-7.

    Exercises: CNN Crack Detection Lab and Salt Picking Lab

    YouTube: Overfitting, Validation, and Regularization. Visualizing and Understanding CNN.

    Day 4

  • Support Vector Machines
  • SVM Migration Noise Elimination in Radon Domain.
  • SVM Migration Noise Elimination in (x,t) Domain
  • SVM Dispersion Picker in FK Domain,

    • Chapter 8.

    Exercises: SVM, NN, and Logistic Regression Denoising of Migration Images Lab.

    Papers: SVM Radon Migration Reduction, SVM migration Reduction in (x,t),LS-SVM, Analytic SVM Fast Classification of Medical Images, Jing Dispersion Window Picking by SVM, Amr SVM Multiple Elimination,

    YouTube: Support Vector Machines and Kernel Methods.

    Day 4

  • K-Means Clustering
  • Semblance Picking by Clusters,
  • Multiple Elimination in Clusters in (t,p),
  • PCA

    • Chapters 9-10

    Exercises: NMO Velocity Analysis by the K-Means Cluster Lab, Yellowstone PCA Lab, Yellowstone PCA Lab. and Simple PCA Lab.

    Papers:PCA 1982 Geophysics, Semblance Analysis by Cluster Picking, PCA Review. TLE Radiometric paper, and Radiometric Tutorial.

    Days 4-5

  • MD,
  • NNLSM,
  • Sparse Coding,
  • Autoencoders,
  • Sparse & Robust Inversion,
  • FWI+NN,

    • Chapters 11-12.

    Exercise: Sparse Convolutional Coding Lab and Keras Autoencoding Lab.

    Youtube Elad's Sparse Modeling in Image Processing and Deep Learning and Elad's Course on SparseLand.

    Day 5

  • GANS Tutorial.
  • FWI and ML,
  • RNN PPT,
  • GANs PPT,
  • Xiangliang GANS.

    • Chapter 7 in Schuster

    Exercises: None


    Youtube: Reinforcement Learning.

    Day 5

    Review of Latest CNN Applications in Exploration Geophysics.

    .

    Exercise: None