Engineering

This text provides a comprehensive introduction to the mathematical theory of probability, its application to the modeling of random phenomena encountered in electrical and computer engineering, and its uses in making optimal decisions and inferences. Fine meets the needs of engineering students by addressing both highly conceptual mathematical methods and their real-world applications. He offers a sound introduction to the many elements of applied probability – the presentation is thorough, yet does not require a more advanced mathematical background beyond basic integral calculus.

• Wide-ranging view of the occurrences of random phenomena throughout history, especially those of interest in physics and electrical engineering – Demonstrates the wealth of applications to daily life, including natural phenomena such as the weather; social and economic phenomena; physical phenomena derived from quantum and statistical mechanics processes, and engineering applications based on the interactions between environmental and designed system random behaviors.

• Five-step approach to learning – Begins with a qualitative discussion of a concept’s roles, applications, or intended uses, followed by a mathematical definition to formalize the concept. The text then addresses elementary implications of the definition through lemmas and theorems, further enriches the discussion with associated definitions, and finally, illustrates it with worked examples and exercises drawn primarily from electrical engineering.

• Sound perspective on probabilistic reasoning – Balances conceptual understanding of probabilistic reasoning with applied skills in modeling and calculation.

• Organized development of theory and applications – Supplements the formal narrative with set-off comments, examples, and sub-sections to clearly illustrate key concepts.

• Careful ordering of topics – Designed to emphasize the logical relationships between the concepts presented.

• Variety of worked-out examples – Clarify the technical results of concepts under discussion, and provide a useful model for the completion of chapter-ending exercises.

• Numbered and labeled definitions, theorems, lemmas, and corollaries – Designed for easy reference.

Contents

(Note: All chapters begin with Purpose, Background, and Organization, and conclude with a Summary and Exercises.)

Sanity Checks — simple, quick ways of checking the plausibility of an answer are inserted in various places in the text.

Chapter 1: Background, Events, and Data

Brief Historical Background to Probability. Important Examples of Random Phenomena. Modeling Random Phenomena I: Random Experiment and Sample Space. Modeling Random Phenomena II: Events and Combinations of Events. Modeling Random Phenomena III: Event Collections and Algebras. Dealing with Chance Data–Statistics. Appendix 1: Sets. Appendix 2: Functions. Appendix 3: Pseudorandom Numbers Generated by Matlab.

 

Chapter 2: Classical Probability

Choosing at Random. Enumeration of Ordered Sequences. Enumeration of Sets: Binomial Coefficients. Application to Entropy and Data Compression. Application to Graphs. Application to Statistical Mechanics. Multinomial Counting. Conditional Classical Probability. Independence in Classical Probability.

 

Chapter 3: Probability Foundations

Hierarchy of Probability Structural Concepts. Interpretations of Probability. Long-run Time Averages and Relative Frequencies. Modeling Random Phenomena IV: Kolmogorov Axioms. Elementary Consequences of the Axioms K0—K4. Boole’s Inequality. Inclusion-Exclusion Principle. Convex Combinations of Probability Measures. Application to Target Detection. Appendix: Proof of Equivalence Theorem.

 

 

Chapter 4: Describing Probability I: Countable

Probability Mass Functions. Application to Modeling English Text Letter Frequencies. Commonly Encountered PMFs. Poisson as a Rare Events Limit of the Binomial.

 

Chapter 5: Describing Probability II: Uncountable and Distributions

Cumulative Distribution Functions. Properties of Univariate CDFs. General Representation and Decomposition of Univariate CDFs. The Empirical CDF.

 

Chapter 6: Describing Probability III: Uncountable and Densities

Probability Density Functions. Representing PMFs by PDFs. Closure Under Convex Combinations. PDFs over Finite Intervals. PDFs over Semi-infinite Intervals. PDFs over all of R. Appendix: Beta Density Unit Normalization.

 

Chapter 7: Multivariate Distribution Functions and Densities

Multivariate or Joint CDFs. Multivariate PDF Representation of Multivariate CDF. Examples of Multivariate PDFs. Application to Reliability.

 

Chapter 8: Functions of Random Variables

Random Variables. Single Input—Single Output (SISO) Functions. Simulation and the Probability Integral Transformation. Multiple Input-Multiple Output (MIMO): n = m. MIMO Applications. MIMO Transformation: m < n.

 

Chapter 9: Expectation and Moments

The Concept of Expectation. Interpretation 1: Mean. Elementary Properties of Expectation. Interpretation 2: Definite Integral. Interpretation 3: Long-run Average Outcome. Interpretation 4: Fair Price. Expectation of a Function of a Random Variable. Convex Functions and Jensen’s Inequality. Moments, Especially Variance. Correlation, Covariance, and the Schwarz Inequality. Linear Dependence and Least Mean Square Estimation. Extensions of Expectation to Vector, Matrix, and Complex-Valued Variables. Correlation and Covariance for Real Random Vectors. Linear Transformation and Synthesis of (Gaussian) Random Vectors. Appendix: Summary of Basic Properties of Expectation. Appendix: Correlation for Complex Random Vectors.

 

Chapter 10: Linear Systems, Signals, and Filtering

Background to Linear Systems. Random Signals and Autocorrelation Functions. Moments of Outputs of Linear Systems. Application to Wiener Filtering. Application to Kalman Filtering.

 

Chapter 11: Discrete Conditional Probability

Conditional Probability. Properties of Discrete Conditional Probability. Multiplication/Sequence Theorem. Total Probability Theorem and Its Applications. Inverting Cause and Effect: Bayes’ Theorem and Its Applications. Three Problems that Challenge Our Intuition. Appendix: Discrete Conditional Probability Summary.

 

Chapter 12: Mixed Conditional Probability

Conditional Probability: P(A|B) for P(B) = 0. Bivariate Conditional Density. Multivariate Conditional Densities. Applications of Conditional Densities. Useful Families of Conditional Densities. Extensions of the Total Probability Theorem. Applications of the Extended Total Probability Theorem. Extensions of the Bayes’ Theorem. Applications of the Extended Bayes’ Theorem. Appendix: Conditional Probability Summary I. Appendix: Conditional Probability Summary II.

 

Chapter 13: MAP, MLE, and Neyman-Pearson Rules

Binary Decision-Making Setup. Minimum Error Probability Design: MAP Rule. Hypothesis Testing. Additive Measurement Noise.

 

Chapter 14: Independence

Independent Pairs of Events. Mutually Independent Events. Independent Random Variables and Product Factorization. Derivations of Two Probability Models. Example of Maxima and Minima of Random Variables. Expectation and Independence. Application to Estimation of the Mean and Variance.

 

Chapter 15: Characteristic Functions 

Characteristic Function Definition and Examples. Properties of the Characteristic Function. Relationship Between Characteristic Functions and Moments. Extension to Random Vectors. Linear Transformations and the Multivariate Normal Revisited. Characteristic Functions and Independence. Characteristic Functions of Sums. Central Limit Theorem. Generating Function.

 

Chapter  16: Probability Bounds and Sums

Generalized Chebychev Bounds. Chernoff Bounds. Hoeffding Bounds.

 

Chapter 17: Conditional Expectation

Conditional Expectation Basics. Example of the Multivariate Normal. MMSE Estimation. Two Applications of MMSE Estimation. Application to Kalman Filtering Revisited. Random Sums. Appendix: Conditioning on Event Algebras.

 

Chapter 18: Bayesian Inference

Setup and Notation for Bayesian Decision-Making. Bayes Approach to Decision-Making. Calculating Bayes Rules. Particular Bayes Solutions. Appendix: An Informal Primer on Loss Functions.

 

Chapter 19: Limits and Laws of Large Numbers

Jacob Bernoulli on the Laws of Large Numbers. LLN for Mean Square Convergence. Convergence of Sequences of Functions. Convergence of Sequences of Random Variables. Mutual Convergence. Laws of Large Numbers. An Example of Divergence of Time Averages. Appendix: Proof of Theorem 19.5.1 C1, C2.

 

Chapter 20: Random Processes

Definition of a Random Process. Borel-Cantelli Lemma. Independent Random Variables. Stationary and Ergodic Processes. Markov Chains in Discrete Time. Independent Increment Random Processes. Martingales. Calculus. Wide Sense Stationary Processes and Power Spectral Density. Gaussian Random Process.

 

Bibliography. Index.

Probability/Random Processes [CORE TEXTS] (Electrical and Computing Engineering)

 

View a Sample Chapter PDF:

  

Pearson Higher Education offers special pricing when you choose to package your text with other student resources. If you're interested in creating a cost-saving package for your students, contact your Pearson Higher Education representative for pricing and ordering information.

Pearson Higher Education offers special pricing when you choose to package your text with other student resources. If you're interested in creating a cost-saving package for your students contact your Pearson Higher Education representative.