ICASSP 2015 will continue the tradition of previous ICASSPs and will offer a wide selection of high-quality tutorials on hot topics for the signal processing community.
This year we offer 17 tutorials to cover the diverse interests of the attendees. Each tutorial will be three hours long and will provide an overview of the state of the art of a particular topic by renowned presenters. Supporting material will be distributed.
The tutorials are scheduled before the start of regular conference sessions in five parallel tracks on Sunday afternoon and six parallel tracks on Monday morning and Monday afternoon.
We hope the attendees will enjoy the diverse choice of tutorials.
Daniel P. Palomar
ICASSP 2015 Tutorial Chair
List of Tutorials
Sunday, 19th April, 2015, Afternoon
1.30pm–5pm (30 minutes break for afternoon coffee/tea at 3pm)
T1—Genomic Signal Processing: From Compression to Knowledge Mining
Subject Area: Biomedical signal processing
Presenters: Olgica Milenkovic and Minji Kim (University of Illinois, Urbana-Champaign)
Location: Meeting Room P5, Plaza Level
T2—Computational Networks: A Generalization of Deep Learning Models
Subject Area: Mathematical tools
Presenters: Dong Yu, Mike Seltzer, Kaisheng Yao, Zhiheng Huang and Jasha Droppo (Microsoft Research)
Location: Meeting Room P1, Plaza Level
Fully booked
T3—Random Matrices, Robust Estimation and Applications
Subject Area: Mathematical tools
Presenter: Romain Couillet (Supélec)
Location: Meeting Room P2, Plaza Level
T4—Mixed-Integer Programming in Signal Processing and Communications
Subject Area: Mathematical tools
Presenters: Marius Pesavento (Technische Universität Darmstadt), Marc E. Pfetsch (Technische Universität Darmstadt) and Yong Cheng (NEC Labs)
Location: Meeting Room P3, Plaza Level
T5—Covariance Analysis and Machine Learning Methods for Electronic Trading
Subject Area: Signal processing for finance
Presenters: Ali Akansu (New Jersey Institute of Technology) and Dmitry Malioutov (IBM)
Location: Meeting Room P4, Plaza Level
Monday, 20th April, 2015, Morning
9am–12:30pm (30 minutes break for morning coffee/tea at 10:30am)
T6—Signal Processing Tools for Big Data Analytics
Subject Area: Big data
Presenters: Georgios B. Giannakis (University of Minnesota), Konstantinos Slavakis (University of Minnesota) and Gonzalo Mateos (University of Rochester)
Location: Meeting Room P1, Plaza Level
Fully booked
T7—Introduction to Signal Processing and Optimization problems in the Smart Electric Power Grid Networks
Subject Area: Signal processing for power & energy
Presenter: Anna Scaglione (Arizona State University)
Location: Meeting Room P4, Plaza Level
T8—Auralization for Architectural Acoustics, Virtual Reality and Computer Games: from Physical to Perceptual Rendering of Dynamic Sound Scenes
Subject Area: Audio signal processing
Presenters: Enzo De Sena (Katholieke Universiteit Leuven), Zoran Cvetkovic (King’s College London) and Julus O. Smith (Stanford University)
Location: Meeting Room M1, Mezzanine Level
T9—Signal Processing for Cochlear Implants
Subject Area: Audio signal processing/biomedical signal processing
Presenters: Oldooz Hazrati (University of Texas at Dallas), John H. L. Hansen (University of Texas at Dallas), Brett A. Swanson (Cochlear Ltd. Australia) and Michael Goorevich (Cochlear Ltd. Australia)
Location: Meeting Room P2, Plaza Level
T10—Compressive Covariance Sensing
Subject Area: Sparsity techniques
Presenters: Geert Leus (Delft University of Technology), Zhi Tian (George Mason University) and Daniel Romero (University of Vigo)
Location: Meeting Room P3, Plaza Level
Less than 20 places left
T11—Over-The-Horizon Radar: Fundamental Principles, Adaptive Processing and Emerging Applications
Subject Area: Radar signal processing
Presenter: Giuseppe A. Fabrizio (Defence Science & Technology Organisation, Australia)
Location: Meeting Room P5, Plaza Level
Monday, 20th April, 2015, Afternoon
1:30pm–5pm (30 minutes break for afternoon coffee/tea at 3pm)
T12—Convex Optimization for Big Data
Subject Area: Big data
Presenters: Volkan Cevher (EPFL), Mario Figueiredo (University of Lisbon), Mark Schmidt (University of British Columbia) and Quoc Tran-Dinh (EPFL)
Location: Meeting Room P1, Plaza Level
T13—Adaptation, Learning, and Optimization over Networks
Subject Area: Network distributed signal processing
Presenter: Ali H. Sayed (University of California, Los Angeles)
Location: Meeting Room P3, Plaza Level
T14— Imaging and Calibration for Aperture Array Radio Telescopes
Subject Area: Image processing
Presenters: Amir Leshem (Bar-Ilan University) and Stefan J. Wijnholds (Netherlands Institute for Radio Astronomy)
Location: Meeting Room P4, Plaza Level
T15—Perceptual Metrics for Image and Video Quality in a Broader Context: From Perceptual Transparency to Structural Equivalence
Subject Area: Image processing
Presenters: Sheila S. Hemami (Northeastern University) and Thrasyvoulos N. Pappas (Northwestern University)
Location: Meeting Room P5, Plaza Level
T16—Beyond Randomness: Sparse Signal Processing in Practice
Subject Area: Sparsity techniques
Presenters: Waheed U. Bajwa (Rutgers University) and Marco F. Duarte (University of Massachusetts, Amherst)
Location: Meeting Room M1, Mezzanine Level
T17—Adaptive Learning for Model-Based Blind Source Separation
Subject Area: Audio signal processing
Presenter: Jen-Tzung Chien (National Chiao Tung University)
Location: Meeting Room P2, Plaza Level
Less than 20 places left
Abstracts of Tutorials
T1—Genomic Signal Processing: From Compression to Knowledge Mining
Subject Area: Biomedical signal processing
Presenters: Olgica Milenkovic and Minji Kim (University of Illinois, Urbana-Champaign)
Summary
The tutorial will cover the following emerging topics in bioinformatics, genomic data processing, machine learning and information theory.
Tutorial participants will be given access to the presentation slides, software and data repository links and selected publications online. Paper copies of the slides will not be distributed.
Outline
1. Genomic data acquisition, formatting and database storage. Sequencing technologies, FASTA, FASTQ, SAM/BAM file formats, genomic and metagenomic sequence repositories and databases.
2. Software demonstration: NCBI Sequence Read Archive, UCSD Genome Browser
3. Alignment and Assembly. Computational aspects of alignment, including indexing, transform coding and dynamic programing. Assembly algorithms based on de Bruijn graphs and greedy overlap-layout-consensus methods. Statistical error-correction of reads as part of assembly pre-processing, including the QUAKE algorithm (if time permits).
4. Software demonstration: Bowtie2, BLAST, IDBA, SOAPdenovo, Velvet, Quake
5. Genomic data compression: raw-read and reference based compression. The CRAM, SCALCE, smallWig, MCUIUC (MetaCRAM) algorithms. Golomb and related codes. Compression of RNASeq data. Compressive genomics.
6. Software demonstration: CRAM, SCALCE, Quip, MFCompress, MCUIUC, smallWig
7. Genomic data rearrangements and gene prioritization. Reversal (R), translocation (T), double-cut-and-join (DCJ) distances between DNA sequences. Inference of genome breakages and sequence evolution via R,T, and DCL distances. Prioritizing genes via distances from disease genes and rank aggregation.
8. Software demonstration: Endeavour, ToppGene, HyDRA
9. (Optional) Reverse Engineering of Gene Regulatory Networks. Exploiting sparsity information in the reconstruction process.
Biographies
Olgica Milenkovic received her M.Sc. Degree in Mathematics and Ph.D. in Electrical Engineering from the University of Michigan, Ann Arbor, in 2001 and 2002, respectively. From 2002 until 2006, she was with the faculty of the Electrical Engineering Department, University of Colorado, Boulder. In 2007, she joined the University of Illinois, Urbana-Champaign, where she currently holds the title of associate professor. Her research interests lie in the areas of bioinformatics, coding theory, compressive sensing and social sciences. Olgica Milenkovic is a recipient of the NSF Career Award, the DARPA Young Faculty Award, and the Dean’s Excellence in Research Award. In 2012, she was named a Center for Advanced Studies (CAS) Associate, while in 2013, she was awarded the Willett scholarship. From 2006, 2009 and 2011, she served on the editorial board for the IEEE Transactions on Communications, the IEEE Transactions on Signal Processing and the IEEE Transactions on Information Theory, respectively. She was the technical program co-chair of the 2014 International Symposium on Information Theory (ISIT), and the Allerton Conference co-chair in 2013 and 2014.
Minji Kim is a PhD Candidate at the University of Illinois at Urbana-Champaign, in the Electrical and Computer Engineering department. She received her Bachelor of Science in Electrical Engineering and Mathematics (Honors with Distinction) from the University of California at San Diego (UCSD) in 2011. Her research interests are DNA compression, metagenomics, and gene prioritization. She is a recipient of the NSF Graduate Research Fellowship, Gordon Scholarship, UCSD Undergraduate Research Conference Award, Jacobs School of Engineering Outstanding Leadership Award. She is a member of Tau Beta Pi and the IEEE.
T2—Computational Networks: A Generalization of Deep Learning Models
Subject Area: Mathematical tools
Presenters: Dong Yu, Mike Seltzer, Kaisheng Yao, Zhiheng Huang and Jasha Droppo (Microsoft Research)
Fully booked
Summary
Many popular machine-learning models for prediction and classification, such as deep neural networks (DNNs), convolutional neural networks (CNNs), recurrent neural networks (RNNs), long short-term memory (LSTM) and maximum-entropy models can be described as a series of computation steps. Such models can be represented using a structure known as a computational network (CN). A computational network expresses a model’s operation as a graph, where leaf nodes represent input values or learnable parameters and parent nodes represent basic computations, such as sum, multiplication or logarithm. Arbitrarily complex computations can be performed using a sequence of such nodes.
The computational network toolkit (CNTK) is a general purpose C++-based machine-learning toolkit for training and evaluating any model that can be described using a computational network. The core of CNTK is an internal representation of the network which provides two key methods: Evaluate and ComputeGradient. Evaluate computes the value of a node given its inputs and ComputeGradient computes the gradient of a node with respect to its inputs. These methods are intelligently scheduled by an execution engine that supports processing on a CPU or a GPU. CNTK uses a network-description language that enables the user to specify the desired model structure, sequence of operations, objective function and parameters to be learned. A network builder reads the network description and creates a computational network. Existing models can also be copied, modified, merged or otherwise manipulated using a built-in model-editing language.
In this tutorial, we will first introduce the prevailing deep-learning models such as DNNs, CNNs, RNNs and LSTM. We will discuss their strengths and weaknesses as well as the difficulty of implementing them as separate models. We will then show how these standard models as well as models of arbitrary topology, connectivity and recurrence can be expressed and constructed as a CN. We describe the key algorithms used in evaluating and training CNs. After that, we introduce the CNTK with focus on how to exploit and extend it to accelerate your own deep-learning research. Specific examples from well known tasks in speech and language processing will also be shown including acoustic modeling, language modeling and spoken-language understanding.
Materials provided
Tutorial slides, CNTK Source code, documentation and examples.
Outline
1. Prevailing Deep Learning Models: An Introduction (30 mins)
a. DNN
b. CNN
c. RNN
d. LSTM
2. Computational Network: A Unified Framework for Models Expressible as Functions (50 mins)
a. Generalization of Deep Learning Models
b. CN Representation, Forward Computation, and Gradient Computation
c. Computation Nodes
d. Recurrent Networks
3. Computational Network Toolkit: A Generic Toolkit for Building CNs (50 mins)
a. CNTK Architecture
b. Running CNTK
c. I/O using Data Readers and Writers
d. Network Builder and Definition Language
e. Optimization Algorithms
f. Model Editing Language
g. Extending the functionality of CNTK
4. Examples: Acoustic Model, Language Model, Spoken Language Understanding (30 mins)
a. DNN-HMM AM example
b. LSTM-HMM AM example
c. Prediction based AM example
d. Class-based RNN LM example
e. LSTM LM example
f. LSTM SLU example
Biographies
Dong Yu is a principal researcher at Microsoft Research—Speech and Dialog Research Group. He holds a Ph.D. degree in computer science from University of Idaho, an MS degree in computer science from Indiana University at Bloomington, an MS degree in electrical engineering from Chinese Academy of Sciences, and a BS degree (with honor) in electrical engineering from Zhejiang University (China). His current research interests include speech processing, robust speech recognition, discriminative training and machine learning. He has published over 140 papers in these areas and is the inventor/co-inventor of more than 50 granted/pending patents. His work on context-dependent deep neural network hidden Markov model (CD-DNN-HMM) has helped to shape the new direction on large vocabulary speech recognition research and was recognized by the IEEE SPS 2013 best paper award. Most recently, he has focused on applying computational networks, a generalization of many neural network models, to speech recognition. Dr. Dong Yu is currently serving as a member of the IEEE Speech and Language Processing Technical Committee (2013–) and an associate editor of IEEE Transactions on Audio, Speech, and Language Processing (2011–). He has served as an Associate Editor of IEEE Signal Processing Magazine (2008–2011) and the lead Guest Editor of IEEE Transactions on Audio, Speech, and Language Processing Special Issue on Deep Learning for Speech and Language Processing (2010–2011).
Mike Seltzer received the Sc.B. degree with honors from Brown University in 1996, and M.S. and Ph.D. degrees from Carnegie Mellon University in 2000 and 2003, respectively, all in electrical engineering. From 1998 to 2003, he was a member of the Robust Speech Recognition Group at Carnegie Mellon University. Since 2003, Dr. Seltzer has been a member of the Speech Research Group at Microsoft Research, where he is currently a Senior Researcher. In 2006, he was awarded the IEEE SPS Best Young Author paper award for his work on microphone array processing for speech recognition. While at Microsoft, Dr. Seltzer has made scientific contributions in noise robustness and speech enhancement, and his algorithms are used in several Microsoft products including Bing Voice Search, Windows Phone, Windows Automotive and Windows Live Messenger. He is currently a member of the IEEE Speech and Language Technical Committee (SLTC) and from 2006–2008, he was Editor-in-Chief of the SLTC e-Newsletter. From 2009–2011, he was an Associate Editor of the IEEE Transactions on Audio, Speech, and Language Processing. His current interests include speech recognition in adverse environments, acoustic modeling and adaptation, neural networks, microphone arrays and machine learning for speech and audio applications.
Kaisheng Yao is a senior research engineer at Microsoft Research. He received his Ph.D. degree in Electrical Engineering in a joint program of Tsinghua University, China, and Hong Kong University of Science and Technology in 2000. From 2000 to 2002, he worked as an invited researcher at Advanced Telecommunication Research Lab in Japan. From 2002 to 2004, he was a post-doc researcher at Institute for Neural Computation at University of California at San Diego. From 2004 to 2008, he was with Texas Instruments. He joined Microsoft in 2008.
He has been active in both research and development areas including natural language understanding, speech recognition, machine learning and speech signal processing. He has published more than 50 papers in these areas and is the inventor/co-inventor of more than 20 granted/pending patents. At Microsoft, he has helped in shipping products such as voice search and Xbox. His current research and development interests are in the areas of deep learning using recurrent neural networks and its applications to natural-language understanding, query understanding and speech processing.
Zhiheng Huang obtained his MSc and PhD degrees in Artificial Intelligence from University of Edinburgh at UK in 2002 and 2006 respectively. He then moved to University of California at Berkeley in 2006 for postdoc research on open domain question answering. From 2009/12 he worked as a scientist at Yahoo! Labs on query linguistic analysis and web search. From 2012/4, he was a senior scientist at Microsoft working on speech recognition. From 2014/11, he has been a senior researcher at Baidu working on deep learning. His research interests include natural language processing, machine learning and speech recognition.
Jasha Droppo is a senior researcher at Microsoft Research. He received the B.S. degree in electrical engineering (with honors) from Gonzaga University, Spokane, WA, in 1994 and the M.S. and Ph.D. degrees in electrical engineering from the University of Washington, Seattle, in 1996 and 2000, respectively. At the University of Washington, he helped to develop and promote a discrete theory for time-frequency representations of audio signals, with a focus on speech recognition. He is best known for his research in robust speech recognition, including algorithms for speech signal enhancement, model-based speech feature enhancement, robust speech features, model-based adaptation and noise tracking. His current interests include the use of neural networks in acoustic modeling and the application of large data and general machine learning algorithms to previously hand-authored speech recognition components.
T3—Random Matrices, Robust Estimation and Applications
Subject Area: Mathematical tools
Presenter: Romain Couillet (Supélec)
Summary
For the last ten years, random matrix theory (RMT) has produced significant results in signal processing, by providing a strikingly different approach to many classical problems in multivariate inference. This has led to improved signal detection and DoA estimation methods for array processing, which are now well known in the SP literature. As a common denominator, these works handle random matrices of the sample covariance matrix type, already known since the early works of Marčenko and Pastur in 1967 and then Bai, Silverstein and Choi in 1995. More recently, since 2010 mostly, new considerations of random matrix applications to signal processing have appeared which are based on the study of more involved random matrix structures than sample covariance matrices. In particular, recent works on random matrix applications to robust statistics now allow for a clear understanding of the behavior of robust scatter estimates as well as robust regressors in the not-so-large sample regime. These works have led in particular to the introduction of novel methods for statistical inference in the presence of outliers or impulsive noise (e.g., introduction of novel robust and RMT-compliant MUSIC algorithm derivatives for DoA estimation).
This tutorial introduces (i) the basic methods of random matrix theory for signal processing along with their applications to signal detection, power estimation and DoA estimation and (ii) an introduction to the recent and future research directions in random matrix applications to robust statistics, which we believe constitutes the bridgehead for further extensions of random matrix theory to a wider signal processing application (and even machine learning) scope. Application-wise, the tutorial will cover the generalization of classical array-processing methods to (possibly) large-dimensional and impulsive systems, portfolio optimization in statistical finance, as well as biological considerations based on robust correlation-matrix estimation for applications such as vaccine designs, etc.
The main objectives of the tutorial are both to get attendees acquainted to the now well-established random matrix methods for sample covariance matrices but also to provide them with an outlook on the future challenges and promising breakthroughs expected in the near to far future. For the former aspect, the tutorial will proceed in a methodological way by providing intuitions and sketches of proofs that will help the attendees feel at ease with random matrix tools. The latter aspect, for which unifying tools still do not exist, will take the form of an exposition of recent theoretical results and their applications, with a short presentation of the main (sometimes various) ideas behind the proofs, and will be concluded by open questions. It is expected that this tutorial, unlike past ones on the same topic, would gather a wider audience on topics closer to their own concerns.
Outline
1. Basics of Random Matrix Theory for Sample Covariance Matrices:
a. the Stieltjes transform method,
b. limiting spectrum analysis,
c. G-estimation,
d. extreme eigenvalues (Tracy-Widom law) and
e. spiked models.
2. Applications to Signal Sensing and Array Processing:
a. eigen-based detection,
b. G-MUSIC,
c. spiked G-estimation and
d. estimation in unknown noise environment.
3. Robust Estimation and Random Matrices:
a. robust scatter estimates,
b. robust regressors and
c. robust shrinkage.
4. Recent Results and Future Directions:
a. robust array processing (robust G-MUSIC),
b. optimal shrinkage (portfolio optimization),
c. CLT for robust statistics,
d. applications to correlation estimates in statistical biology, etc.
Biography
Romain Couillet received his MSc in Mobile Communications at the Eurecom Institute and his MSc in Communication Systems in Telecom ParisTech, France in 2007. From 2007 to 2010, he worked with ST-Ericsson as an Algorithm Development Engineer on the Long Term Evolution Advanced project, where he prepared his PhD with Supélec, France, which he graduated in November 2010. He is currently an assistant professor in the Telecommunication department of SUPÉLEC, France. His research topics are in information theory, signal processing, complex systems and random matrix theory. He is a co-author of the book Random matrix methods for wireless communications, Cambridge University Press, 2011.
T4—Mixed-Integer Programming in Signal Processing and Communications
Subject Area: Mathematical tools
Presenters: Marius Pesavento (Technische Universität Darmstadt), Marc E. Pfetsch (Technische Universität Darmstadt) and Yong Cheng (NEC Labs)
Summary
There exists a large variety of applications, for instance in estimation and detection as well as network optimization, that involve both integer (discrete) decision-making and the optimization of continuous parameters. The integer decision-making requirements usually stem from the nature of the problem. In many applications, the physical quantities to be optimized are naturally undividable. Think for example of a cellular network in which a subset of users needs to be selected for transmission. For a given user, a connection is either established or not. In source localization, the number of sources to be estimated is an integer countable quantity that needs to be optimally selected. Or consider a multi-hop sensor network where the number of hops needs to be minimized to achieve minimum latency services, to mention just a few examples. In other applications, integer decision-making is imposed by technical standards. For example, in cellular communication networks built on the current 3GPP standard LTE-Advanced, a large variety of cellular resources and parameters have to be optimized that conventionally in academic research are often treated as continuous. One interesting example is the design of optimal beamformers in multiuser communication systems. It is well established that optimal beamforming vectors can theoretically be computed with low computational complexity, e.g., in closed form or from the solution of a convex optimization problem. However, due to signaling requirements, the LTE standard admits beamforming vectors that are selected from a codebook of predefined candidate beamformers. This involves integer decision-making. Similarly, the data rate at which a user in the network is served depends on the modulation scheme (QPSK, 16-QAM, 64-QAM, etc.) as well as the block size of channel-coding scheme. Thus, rate adaptation in cellular systems generally involves mixed-integer optimization. Another prominent example is MIMO detection and channel decoding. Here integer decision-making is involved in the sense that the detected symbols are confined to the alphabet of constellation symbols or the codewords defined in the communication standard.
Due to the coupling of the decision variables, mixed-integer optimization problems are often of combinatorial nature and exhibit a complexity that grows exponentially with the problem dimension. The computational complexity required to exhaustively test all combinations becomes prohibitive even if the problem size is medium. A popular and straightforward approach that is often considered in practice is to model the integer decision variables as continuous and then quantize the solution obtained from the continuous problem. However, despite its simplicity, this approach, commonly referred to as continuous relaxation, leads in most of the cases to highly suboptimal solutions and often infeasible points of the original mixed-integer problem.
The branch-and-cut algorithm and its variants represent a standard framework for handling mixed-integer programs. Branch-and-cut algorithms are particularly successful in reducing the complexity of the combinatorial search if they are customized to the particular problem under consideration. This requires the development of smart reformulations of the original problem that exhibits tight continuous relaxations, i.e., problem reformulations which after relaxation of some of the integer variables the continuous domain provide close bounds for the original problem. In recent years, a number of problem customization concepts have been developed to accomplish this task, such as the lifting technique in which additional variables are introduced to the problem, the concept of cuts which consists in the introduction of additional redundant constraints that tighten the continuous relaxations, as well as the big-M method which, e.g., allows to avoid (non-convex) bi-linear and tri-linear expressions in the continuous relaxations. While the branch-and-cut method provides a universal framework for a large variety of applications and mixed-integer problems it generally suffers from slow and non-deterministic convergence. Provably optimal solutions can only be obtained for small problem dimensions. In particular applications, it is, however, despite the combinatorial problem structure, possible to overcome the limitations of the standard techniques and to transform the problem into equivalent formulations with decoupled binary variables. One interesting example is the codebook-based beamforming in cellular downlink networks for which a particular form of uplink-downlink duality result can be established to convert the originally combinatorial problem into a much simpler non-combinatorial mixed-integer problem. Recent works have shown increasing interest in developing efficient heuristics for computing approximate solutions in reasonable run-time. In many cases the underlying problem structure can be exploited to develop tight approximations that often lead to significant savings in the computational complexity and close-to-optimal feasible solutions.
The objective of this tutorial is to introduce a general framework for addressing mixed-integer linear and nonlinear programs and to provide the audience with insight and guidelines how to use existing tools for solving this difficult class of problems. We start the tutorial revising well-known example applications widely known in signal processing and communication applications where mixed-integer decision-making plays an important role. Based on these examples we will discuss important properties commonly encountered in mixed-integer programming and introduce the underlying theoretical tools and methods that can be applied to compute optimal solutions. In this context we will revise the branch-and-cut algorithm and its variants. We will provide examples for standard customization techniques that can be applied to a large variety of problems. We will outline basic steps that can be used in the theoretical analysis to prove that particular reformulations are preferable over others. After covering the basic concepts of mixed-integer programming, we will draw our attention to the software tools and solvers that are available for free, academic or commercial use. We will discuss their distinct features and provide references and benchmark results regarding their respective performances. This shall enable the audience to find a quick starting point for their own problem treatment and code generation. The third part of the tutorial will then focus on advanced applications and recent research results. We will demonstrate with common examples how problem structures can be exploited to develop smart heuristics that can be used to obtain close-to-optimal solutions in just a fraction of time required for computing certified optimal solutions. We conclude the tutorial with some advanced exemplary applications in which particularly elegant algorithmic procedures have been developed that lead beyond branch-and-cut and that allow the design algorithms that reach proven global optimality with polynomial complexity.
Outline
1. Basic Concepts
a. Overview and Applications
b. Introduction: Basic concepts (Examples 1 and 2)
i. branch-and-bound, continuous relaxation
ii. cuts, Big-M, branch-and-cut
iii. branching priorities, branching directions
2. Software Tools
3. Application Examples
a. Example 3: Adaptive Coding and Modulation in Cellular Networks
b. Example 4: Codebook-based Beamforming
c. Example 5: Compressive sensing and mixed-integer programming
Biographies
Marius Pesavento received the Dipl.-Ing. and M. Eng. degrees from Ruhr-Universität Bochum, Germany, and McMaster University, Hamilton, ON, Canada, in 1999 and 2000, respectively, and in 2005 the Dr.-Ing. degree in Electrical Engineering from Ruhr-Universität Bochum, Germany. Between 2005 and 2007, he was a Research Engineer at FAG Industrial Services GmbH, Aachen, Germany. From 2007 to 2009, he was the Director of the Signal Processing Section at mimoOn GmbH, Duisburg, Germany. He became Assistant Professor for Robust Signal Processing in 2010 and Full Professor for Communication Systems in 2013 at the Department of Electrical Engineering and Information Technology, Technische Universität Darmstadt, Darmstadt, Germany. His research interests are in the area of robust signal processing and adaptive beamforming, high-resolution sensor array processing, transceiver design for cognitive radio systems, cooperative communications in relay networks, MIMO and multiantenna communications, space-time coding, multiuser and multicarrier wireless communication systems, convex optimization and mixed-integer programming for signal processing and communications, statistical signal processing, spectral analysis, parameter estimation and detection theory. Dr. Pesavento was a recipient of the 2003 ITG/VDE Best Paper Award, the 2005 Young Author Best Paper Award of the IEEE Transactions on Signal Processing, and the 2010 Best Paper Award of the CROWNCOM conference. He is a member of the Editorial board of the EURASIP Signal Processing Journal, Associate Editor for the IEEE Transactions on Signal Processing, and member of the Sensor Array and Multichannel (SAM) Technical Committee of the IEEE Signal Processing Society (SPS). He served as Technical Co-chair at the IEEE Sensor Array and Multichannel Workshop (SAM 2014).
Marc E. Pfetsch obtained a Diploma degree in mathematics from the University of Heidelberg, Germany, in 1997. He received a PhD degree in mathematics in 2002 and the Habilitation degree in 2008 from TU Berlin, Germany. From 05–07/2007, he was a visiting researcher at the Istituto di Analisi dei Sistemi ed Informatica (IASI), Rome. From 2008 to 2012, he was a Full Professor for Mathematical Optimization at TU Braunschweig, Germany. Since 04/2012, Marc Pfetsch has been a Full Professor for Discrete Optimization at TU Darmstadt, Germany. His research interests are mostly in discrete optimization, in particular symmetry in integer programs, compressed sensing and algorithms for mixed-integer programs. He is one of the developers of the branch-and-cut-and-price framework SCIP, which is currently the fastest academic solver for mixed-integer programs. He is involved as a principal investigator in three projects of two collaborative research centres together with mechanical engineering and was part in a large, industry-funded project on the optimization of natural gas transportation. Marc Pfetsch was member of the program committee of CPAIOR 2011 and 2012. He is the web editor of the Mathematical Optimization Society (MOS) and is associate editor of Operations Research Letters.
Yong Cheng received the B.Eng. (1st class honors), M.Phil., and Ph.D. degrees from Zhejiang University, Hangzhou, P. R. China, the Hong Kong University of Science and Technology, Hong Kong, and Technische Universität Darmstadt, Darmstadt, Germany, in 2006, 2010 and 2013, respectively, all in Electrical Engineering. From Apr. 2009 to Jun. 2010, he worked in the Huawei-HKUST Innovation Laboratory, Hong Kong. He currently works as a Research Scientist at NEC Laboratories Europe, Heidelberg, Germany. His research interests mainly include mixed-integer programming and convex optimization in signal processing and wireless communications, multiple-antenna techniques in LTE/LTE-Advanced systems, as well as optimized discrete resource allocation and coordinated multipoint (CoMP) processing in heterogeneous and small cell networks (HetSNets).
T5—Covariance Analysis and Machine Learning Methods for Electronic Trading
Subject Area: Signal processing for finance
Presenters: Ali Akansu (New Jersey Institute of Technology) and Dmitry Malioutov (IBM)
Summary
Financial markets present a very rich and diverse application area for signal processing techniques. One of the critical challenges for both risk management, portfolio management and design of trading strategies is to understand the statistical dependence and interaction among the high-dimensional time-series. The techniques that are used for these problems span across multiple areas of signal processing and change dramatically depending on the trading frequency, ranging from covariance modelling to short-term prediction based on market microstructure.
High performance computing and DSP technologies facilitating implementation of sophisticated analysis and modelling of market data in real-time and high frequencies have transformed the financial industry for the last few years. This exponentially increasing computational power with cost efficiency brought in once hard-to-implement machine-learning tools for the service of financial sector.
This tutorial introduces the essentials of covariance-analysis and machine-learning methods used in quantitative trading. It describes the state of the art in eigenanalysis, eigenportfolios and statistical arbitrage, sparse representation techniques, including sparse Markowitz portfolios and sparse covariance modelling and machine-learning methods for electronic trading. It also discusses some ongoing research problems and offers insights and signal-processing perspectives in these relatively new fields.
Outline
1. Financial Markets and Signals
2. Covariability and Covariance
3. Trading Frequency and the Epps Effect
4. Eigenanalysis
5. Eigenportfolios and Statistical Arbitrage
6. Sparse Representations
7. Sparse Portfolio Analysis and Sparse Covariance Modeling
8. Machine Learning Methods in Finance
9. Electronic Trading Strategies
10. Discussions
Biographies
Ali N. Akansu (IEEE Fellow) received the B.S. degree from the Technical University of Istanbul, Turkey, the M.S. and Ph.D. degrees from the Polytechnic University, Brooklyn, New York, all in Electrical Engineering. Since 1987, he has been with the New Jersey Institute of Technology, where he is a Professor of Electrical and Computer Engineering. Dr. Akansu has administered and managed a number of research programs and product development projects in academia and private sector, funded by the State & Federal Government agencies, and industry. He was a Founding Director of the New Jersey Center for Multimedia Research (NJCMR) between 1996–2000, and NSF Research Center (IUCRC) for Digital Video between 1998–2000. Dr. Akansu was the Vice President for Research and Development of IDT Corporation. He was the founding President and CEO of PixWave, Inc., an IDT subsidiary. He was an academic visitor at David Sarnoff Research Center, IBM T. J. Watson Research Center and at GEC-Marconi Electronic Systems Corp. He was also a Visiting Professor at the Courant Institute of Mathematical Sciences of NYU. He regularly consults to the industry and legal sector.
Dr. Akansu is widely published and lectures frequently on various research and technology topics, guided theses on signals & transforms, and applications in image/video coding, digital communications, Internet multimedia and content security, and financial signal processing. He is a co-author (with R. A. Haddad) of the book Multiresolution Signal Decomposition: Transforms, Subbands and Wavelets, Academic Press, 1992 and 2001 (2nd Ed.), and a co-editor (with M. J. T. Smith) of a book entitled Subband and Wavelet Transforms: Design and Applications, Kluwer, 1996. He is a co-editor of the book (with M. J. Medley) Wavelet, Subband and Block Transforms in Communications and Multimedia, Kluwer, 1999. He is also a co-author of a research monograph (with H. T. Sencar and M. Ramkumar) Data Hiding Fundamentals and Applications: Content Security in Digital Multimedia, Elsevier-Academic Press, 2004.
Dr. Akansu has served as an associate editor of IEEE Transactions on Signal Processing and IEEE Transactions on Multimedia, as a member of the Signal Processing Theory & Methods, and Multimedia Signal Processing technical committees of the IEEE Signal Processing Society. He organized the first Wavelets Conference in the United States in April 1990. He was the technical program chairman of IEEE Digital Signal Processing Workshop 1996, Loen, Norway. He served as a member of the Steering Committee and the Publications Chair of IEEE ICASSP 2000, Istanbul, Turkey. He was the Lead Guest Editor of two special issues of IEEE Trans. on Signal Processing on Theory and Application of Filter Banks and Wavelet Transforms (April 1998), and on Signal Processing for Data Hiding in Digital Media and Secure Content Delivery (June 2003). He was also the Lead Guest Editor of the issue of the IEEE Journal of Special Topics on Signal Processing on Signal Processing Methods in Finance and Electronic Trading (August 2012).
Dmitry Malioutov is a research staff member in the Machine Learning group in the Business Analytics and Mathematical Sciences (BAMS) department at the IBM T. J. Watson Research Center, Yorktown Heights, NY. Prior to joining IBM, Dmitry had spent several years as an applied researcher in high-frequency trading in DRW Trading, Chicago, and as a postdoctoral researcher in Microsoft Research, UK. Dmitry received the Ph.D. and the S.M. degrees in EECS from MIT. He has received the IEEE Signal Processing Society best paper award in 2010, and IEEE ICASSP best student paper award in 2006, and the MIT presidential fellowship. Dmitry serves on the machine learning in signal processing committee (MLSP) and as an associate editor for the IEEE Transactions on Signal Processing.
His research interests include inference and learning in graphical models, message passing algorithms; Sparse signal representation; Sensor array source localization; Statistical risk modeling, robust covariance and joint dependence modeling; portfolio optimization. More generally: statistical signal processing, machine learning and convex optimization; applications in quantitative finance.
T6—Signal Processing Tools for Big Data Analytics
Subject Area: Big data
Presenters: Georgios B. Giannakis (University of Minnesota), Konstantinos Slavakis (University of Minnesota) and Gonzalo Mateos (University of Rochester)
Fully booked
Summary
We live in an era of data deluge. Pervasive sensors collect massive amounts of information on every bit of our lives, churning out enormous streams of raw data in various formats. Mining information from unprecedented volumes of data promises to limit the spread of epidemics and diseases, identify trends in financial markets, learn the dynamics of emergent social-computational systems and also protect critical infrastructure including the smart grid and the Internet’s backbone network. While Big Data can be definitely perceived as a big blessing, big challenges also arise with large-scale datasets. The sheer volume of data makes it often impossible to run analytics using a central processor and storage, and distributed processing with parallelized multi-processors is preferred while the data themselves are stored in the cloud. As many sources continuously generate data in real time, analytics must often be performed “on-the-fly” and without an opportunity to revisit past entries. Due to their disparate origins, the resultant datasets are often incomplete and include a sizable portion of missing entries. In addition, massive datasets are noisy, prone to outliers and vulnerable to cyber-attacks. These effects are amplified if the acquisition and transportation cost per datum is driven to a minimum. Overall, Big Data present challenges in which resources such as time, space and energy are intertwined in complex ways with data resources. Given these challenges, ample signal processing (SP) opportunities arise. This tutorial seeks to provide an overview of ongoing research in novel models applicable to a wide range of Big Data analytics problems, as well as algorithms and architectures to handle the practical challenges, while revealing fundamental limits and insights on the mathematical trade-offs involved.
Outline
I. Introduction, motivation and context (20 mins.)
II. Theoretical and statistical foundations for Big Data Analytics (1 hr.)
a) High-dimensional statistical SP and succinct data representations;
i) Compressive sampling, sparsity, and (non-linear) dimensionality reduction
ii) Low-rank models, matrix completion, and regularization for underdetermined problems
b) Robust approaches to coping with outliers and missing data;
c) Big tensor data models and factorizations; streaming analytics
III. Algorithmic advances for mining massive datasets (45 mins.)
a) Scalable, online, and decentralized learning and optimization;
b) Randomized algorithms for very large matrix, graph, and regression problems;
c) Convergence analysis, computational complexity, and performance
IV. Random sampling and consensus ideas for Big Data Analytics (45 mins.)
a) Sketching for (non) parametric regression and dynamic data tracking;
b) Classification and clustering large-scale, high-dimensional datasets;
V. Concluding remarks (10 mins.)
Biographies
G. B. Giannakis (IEEE Fellow) received his Diploma in Electrical Engr. from the Ntl. Tech. Univ. of Athens, Greece, 1981. From 1982 to 1986, he was with the Univ. of Southern California (USC), where he received his MSc. in Electrical Engineering, 1983, MSc. in Mathematics, 1986, and Ph.D. in Electrical Engr., 1986. Since 1999, he has been a professor with the Univ. of Minnesota, where he now holds an ADC Chair in Wireless Telecommunications in the ECE Department and serves as director of the Digital Technology Center.
His general interests span the areas of communications, networking and statistical signal processing—subjects on which he has published more than 370 journal papers, 630 conference papers, 21 book chapters, two edited books and two research monographs (h-index 110). Current research focuses on big data analytics, wireless cognitive radios, renewable energy, power grid, gene-regulatory, and social networks. He is the (co-) inventor of 23 patents issued, and the (co-) recipient of 8 best paper awards from the IEEE Signal Processing (SP) and Communications Societies, including the G. Marconi Prize Paper Award in Wireless Communications. He also received Technical Achievement Awards from the SP Society (2000), from EURASIP (2005), a Young Faculty Teaching Award, the G. W. Taylor Award for Distinguished Research from the University of Minnesota, and the IEEE Fourier Technical Field Award (2014). He is a Fellow of EURASIP, and has served the IEEE in a number of posts, including that of a Distinguished Lecturer for the IEEE-SP Society.
Konstantinos Slavakis received the M.Eng. and Ph.D. degrees in Electrical and Electronic Engineering from Tokyo Institute of Technology, (TokyoTech), Japan, in 1999 and 2002, respectively. He has been a Japanese Government Scholar, a JSPS Postdoc at TokyoTech, and a PostDoc with the Dept. of Informatics and Telecommunications, University of Athens, Greece. He served as an Assistant Professor in the Dept. of Telecommunications and Informatics, at the University of Peloponnese, Greece, and he is currently a Research Associate Professor in the Dept. of ECE, and the Digital Technology Center, University of Minnesota.
Research interests include signal processing, machine learning and big data analytics. He served IEEE Trans. on Signal Processing (TSP) as Associate Editor (2009-2013) and he is currently a Senior Area Editor of IEEE TSP since 2010. He has also delivered tutorials talks in ICASSP’12 and ICASSP’14.
Gonzalo Mateos was born in Montevideo, Uruguay, in 1982. He received his B.Sc. degree in Electrical Engineering from Universidad de la Republica, Uruguay, in 2005 and the M.Sc. and Ph.D. degrees in Electrical Engineering from the University of Minnesota (UofM), Twin Cities, in 2009 and 2011. From 2004 to 2006, he worked as a Systems Engineer at Asea Brown Boveri (ABB), Uruguay. During the 2013 academic year, he was a visiting scholar with the Computer Science Dept., Carnegie Mellon University. Since 2014, he has been an Assistant Professor with the Department of Electrical and Computer Engineering at the University of Rochester, Rochester, NY. His research interests lie in the areas of statistical learning from Big Data, network science, wireless communications and signal processing. His current research focuses on algorithms, analysis and application of statistical signal processing tools to dynamic network health monitoring, social, power grid and Big Data analytics. Since 2012, he serves on the Editorial Board of the EURASIP Journal on Advances in Signal Processing. He received the Best Student Paper Award at the 13th IEEE Workshop on Signal Processing Advances in Wireless Communications, 2012, held at Cesme, Turkey, and was also a finalist of the Student Paper Contest at the 14th IEEE DSP Workshop, 2011, held at Sedona, Arizona, USA. His doctoral work has been recognized with the 2013 UofM’s Best Dissertation Award (Honorable Mention) across all Physical Sciences and Engineering areas.
T7—Introduction to Signal Processing and Optimization problems in the Smart Electric Power Grid Networks
Subject Area: Signal processing for power & energy
Presenter: Anna Scaglione (Arizona State University)
Summary
In recent years electric power systems are undergoing significant modernization. “Smart Grid” initiatives have flourished, driven in part by policies for incorporating more renewable power and reducing greenhouse gas emissions and, in part, by the need of introducing further economic efficiencies in the way energy resources are used. The expectation is that the Smart Grid will represent a leap forward towards a more flexible, less centralized, more resilient and interoperable design of the grid physical and cyber system. The emerging trend is to have orders of magnitude more data available, and dedicated cloud services to manage energy resources and consumption, using advanced grid status analytics. What are the new models, fundamental tools and new understanding that will be needed to fulfill this vision?
To lay down the foundations, this tutorial will, in its first part, review the basic models for electric power systems, generation, flows and management. This includes the ways sensor information is aggregated and utilized today for grid operations, including the classic sensor regression problems of system state estimation (PSSE), contingency analysis (CA), the optimal power flow (OPF) formulation and its application in electric power markets in the US.
The tutorial will then move to its second part on what are the future trends: a wide-spread use of more powerful synchrophasor technology for sensing the grid state, two-way communications with responsive electric load and distributed generation and dynamic pricing of electricity. The tutorial will explain the need for fast-ramping generation capacity and costly reserves to balance the grid under high penetration levels of intermittent resources and what new are the new design paradigms that would allow electricity demand to respond to economic signals. It is expected that the grid will soon incorporate an access layer to manage the access to power by a set of flexible appliances. Aggregators are the entities that will interact with the wholesale market and manage large populations of flexible demand. The objective is for the Aggregators to utilize an interface that is safe for the grid and economically desirable for the consumers. In the tutorial, we will highlight the design opportunities and challenges in dealing with the modeling problem behind the Aggregator design. Electrical Vehicles will be our case study to illustrate advances in the ex-ante modeling and the on-line scheduling of flexible loads. Open problems and challenges that are relevant to signal processing will be emphasized throughout the tutorial.
Materials provided
The participants will receive a monograph the author is currently writing on “Smart Electric Power Grid Networks” for the Foundations and Trends in Signal Processing Journal. The tutorial and monograph will be a condensed version of an introductory graduate course in demand side management for electric power grids. The non-expert can learn in the first part basic concepts of power systems (a graduate course taught by Anna Scaglione will be condensed to provide the background) and more advance design principles will be instead occupying the remaining part of the tutorial.
Outline
Energy delivery is composed of three flows: power, economic and information flows.
1) Basics of power delivery
– Energy generation
– Energy transmission
– Principles of electrical power transfer and AC power flow model
2) The Market: Economic Dispatch of Electrical Power
– Unit Commitment
– Optimal Power flow
– Demand Response and sustainable power delivery
3) Information systems supporting energy and their vulnerabilities
– Monitoring of the state and data injection attacks
– Forecasting of demand and renewables
– Protective relays and physical cyber attacks
Biographies
Anna Scaglione (IEEE Fellow) is currently Professor at the Ira Fulton School of Engineering at Arizona State University. Prior to ASU, she joined UC Davis in 2008, after leaving Cornell University, Ithaca, NY, where she started as Assistant Professor in 2001 and became Associate Professor in 2006. Prior to joining Cornell she was Assistant Professor in the year 2000–2001, at the University of New Mexico. Dr. Scaglione is a Fellow of the IEEE since 2011 and was honored by both the Signal Processing and the Communication Societies. She is the Editor-in-Chief of the IEEE Signal Processing Letters, and served as Associate Editor for the IEEE Transactions on Wireless Communications from 2002 to 2005, and from 2008 to 2011 in the Editorial Board of the IEEE Transactions on Signal Processing from 2008, where she was Area Editor in 2010–11. She has been general chair of the workshop SPAWC 2005 in the Signal Processing for Communication Committee from 2004 to 2009, has been part of the SmartGridComm steering committee since 2010, and is currently in the Board of Governors of the Signal Processing Society. Dr. Scaglione is the first author of the paper that received the 2000 IEEE Signal Processing Transactions Best Paper Award. She has also received the NSF Career Award in 2002, is co-recipient of the Ellersick Best Paper Award (MILCOM 2005), and co-recipient of the 2013 IEEE Donald G. Fink Prize Paper Award. Her expertise is in the broad area of signal processing for communication systems and networks. Her current research focuses on studying and enabling decentralized learning and signal processing in networks of sensors. Dr. Scaglione also focuses on sensor systems and networking models for the demand side management and reliable energy delivery.
T8—Auralization for Architectural Acoustics, Virtual Reality and Computer Games: from Physical to Perceptual Rendering of Dynamic Sound Scenes
Subject Area: Audio signal processing
Presenters: Enzo De Sena (Katholieke Universiteit Leuven), Zoran Cvetkovic (King’s College London) and Julus O. Smith (Stanford University)
Summary
Auralization is the process of generating the auditory experience of a sound source in a given space. Typically, one or more sound sources radiate into the acoustic space and the acoustic field is sampled by the ears of one or more listeners. Both the source(s) and listener(s) can be moving in the space, and the boundaries of the space itself can be moving (opening doors, etc.). Towards generating this auditory experience, auralization involves two steps: (a) simulation of the desired room acoustics and (b) synthesis of the corresponding sound field in the user space. While both problems can be solved in many ways, in practice one seeks to minimize computational cost and equipment load while maximizing perceived quality of the spatial sound. To that end, both steps should be solved to the extent necessary to satisfy each listener’s perceptual discrimination.
Available room acoustic simulation methods can be classified on a scale that goes from very high computational complexity and high physical accuracy to low computational complexity and reliance on psychoacoustic phenomena. At the former end of the scale are methods such as finite difference time domain (FDTD) and digital waveguide mesh (DWM), which are based on the time and space discretization of the wave equation. Geometric acoustic models, such as beam tracing and the image method, make the simplifying assumption that sound travels as rays. This enables to lower the computational complexity at the cost of a diminished physical accuracy. On the other end of the scale are inexact models that render accurately only the most important perceptual cues. Recently, a method called scattering delay network (SDN) was proposed with this aim. SDNs were shown to render accurately some of the most important perceptual cues while having a computational complexity orders of magnitude lower than physical, geometrical and convolutional methods.
The second step in auralization is to make the simulated room response audible so that the sound source and the associated reflections are perceived in the correct directions. Within this context, existing soundfield recording and reproduction technologies can be used, with the only difference being that both the soundfield and microphones are virtual in this case. Technologies that aim at the physical reconstruction of the soundfield, e.g., higher-order ambisonics (HOA) and wave-field synthesis (WFS), provide approximately accurate solutions to the problem but require a high number of loudspeakers. Systems with low speaker counts must rely on psychoacoustic phenomena instead. The most commonly used psychoacoustic effect within this context is the so-called summing localisation effect, i.e., the effect by which a pair of loudspeakers radiating highly correlated signals result in a single, fused auditory event. The location of each auditory event can be controlled by selecting inter-channel intensity differences and time differences appropriately. Desired localization can be achieved using intensity differences alone (intensity panning), time differences alone (time panning) or a combination of the two (time-intensity panning). While intensity panning is the most commonly used approach, time-intensity panning was recently shown to yield similar localisation performance but with a more graceful degradation away from the centre of the loudspeaker array.
This tutorial will give a detailed overview of the above topics, with greater emphasis on perception-based methods. The intended audience is students and researchers, both from the industry and academia, with a background in digital signal processing and interest in sound field recording, reconstruction and synthesis.
Outline
1. Room Acoustic Synthesis
a. Physical room acoustics models (FDTD, DWM, geometric models)
b. Basics of room acoustics perception
c. Artificial reverberators
d. Digital Waveguide Networks
e. Scattering Delay Networks
2. Reproduction
a. Physical reconstruction methods (HOA, WFS)
b. Basics of psychoacoustics of sound source localization
c. Intensity panning and time panning
d. Equivalent non-coincident microphone arrays
e. Optimal time-intensity panning
f. Effect of time-intensity trading on localisation uncertainty
Biographies
Enzo De Sena received the B.Sc. degree in 2007 and M.Sc. degree (cum laude) in 2009, both from the Università degli Studi di Napoli “Federico II” in Telecommunications Engineering. He obtained his Ph.D. degree in 2013 from King’s College London with a thesis focusing on multichannel audio, room acoustics simulation and microphone array processing. From 2007 to 2009, he collaborated with the Network Research Lab at UCLA. From 2012 until 2013, he was part of the academic staff of the Division of Engineering at King’s College London, as a Teaching Fellow. From August to September, 2013, he was a Visiting Researcher at the Center for Computer Research in Music and Acoustics at Stanford University. Since October 2013, he is an ER Marie Curie fellow with the Department of Electrical Engineering at KU Leuven.
Zoran Cvetković received the Dipl.Ing.El. and Mag.El. degrees from the University of Belgrade, Belgrade, Yugoslavia, in 1989 and 1992, respectively, the M.Phil. degree from Columbia University, New York, in 1993, and the Ph.D. degree in electrical engineering from the University of California, Berkeley, in 1995. He held research positions at EPFL, Lausanne, Switzerland (1996), and at Harvard University, Cambridge, MA (2002–2004). From 1997 to 2002, he was a Member of Technical Staff at AT&T Shannon Laboratory. He is now a Professor in Signal Processing at King’s College London, London, U.K. His research interests are in the broad area of signal processing, ranging from theoretical aspects of signal analysis to applications in audio and speech technologies, and biomedical engineering. He served as an Associate Editor of IEEE Transactions on Signal Processing.
Julius O. Smith received the B.S.E.E. degree in control, circuits and communication from Rice University, Houston, TX, in 1975 and the M.S. and Ph.D. degrees in electrical engineering from Stanford University, Stanford, CA, in 1978 and 1983, respectively. His Ph.D. research was devoted to improved methods for digital filter design and system identification applied to music and audio systems. From 1975 to 1977, he worked in the Signal Processing Department at ESL, Sunnyvale, CA, on systems for digital communications. From 1982 to 1986, he was with the Adaptive Systems Department at Systems Control Technology, Palo Alto, CA, where he worked in the areas of adaptive filtering and spectral estimation. From 1986 to 1991, he was employed at NeXT Computer, Inc., responsible for sound, music and signal processing software for the NeXT computer workstation. After NeXT, he became an Associate Professor at the Center for Computer Research in Music and Acoustics (CCRMA) at Stanford, teaching courses and pursuing research related to signal processing techniques applied to music and audio systems. Continuing this work, he is currently a Professor of Music and, by courtesy, Electrical Engineering at Stanford University. For more information, see http://ccrma.stanford.edu/ ̃jos/.
T9—Signal Processing for Cochlear Implants
Subject Area: Audio signal processing/biomedical signal processing
Presenters: Oldooz Hazrati (University of Texas at Dallas), John H. L. Hansen (University of Texas at Dallas), Brett A. Swanson (Cochlear Ltd. Australia) and Michael Goorevich (Cochlear Ltd. Australia)
Summary
Hearing aids, which serve as the primary solution for hearing loss, amplify incoming sounds and partially restore the hearing ability for individuals with mild to severe hearing loss. However, their success in restoring hearing depends on the availability of an intact inner ear (i.e., functioning inner hair cells, functioning middle ear-malleus, incus, and stapes). Therefore, individuals with profound hearing loss who usually have limited functioning hair cells left receive no benefit with hearing aids. As a solution to this problem, cochlear implants (CI) can be used to partially restore hearing. CIs are good examples of areas where digital signal processing plays a vital role to improve the quality of life for humans. CI users usually perform well in anechoic quiet environments, but speech perception degrades in adverse conditions such as ambient noise, competing talkers and reverberation. Signal processing techniques that have been introduced commercially to address these issues include dual-microphone adaptive beam forming, multi-band gain control, SNR-based noise cancellation, wind-noise reduction and auditory scene classification. There remains a wide range of opportunities for signal processing innovations in this domain.
A second challenge is that CI pitch perception is generally very poor. This limits the enjoyment of music, causes difficulties with tonal languages and makes it harder to segregate competing talkers. Attempts to improve pitch perception by changing the sound processing and stimulation strategy have been largely unsuccessful. This is most likely due to the inability of a CI to reproduce the spatio-temporal neural firing pattern evoked by resolved harmonics in normal hearing.
The aim of this tutorial is to provide an overview of the recent advancements in Signal Processing for Cochlear Implants. No workshops or tutorials have been previously devoted to this field in the past ICASSP conferences, ASRU or SLT workshops. As there has been vast algorithmic advancements in the field of speech and audio processing for Automatic Speech Recognition and Speaker Recognition Systems, an introduction to the signal processing techniques in cochlear implant devices and the available challenges will draw attention of researchers and engineers towards this area, which may result in development of more robust signal processing techniques for CI devices. Although the interest in studying cochlear implants and the challenges in this area has significantly increased recently, there still remain a wide variety of cochlear implant related signal processing needs. Therefore, providing an overview to the field will help researchers to consider the existing issues in their signal processing algorithm developments. Also, the cost of present day CIs is still high, and it is expected that production and implant costs, especially in developing countries, can be reduced with further research advancements in both signal processing and engineering hardware design.
Outline
1. Historical Overview of Signal Processing for Cochlear Implants: Here, we will provide a comprehensive summary of signal processing strategies used in cochlear implant devices from their emergence to date. This section will include audio examples of vocoded speech in order to provide a sense of electrical hearing with CI devices to normal hearing listeners.
2. State of the Art Challenges: In this section, we will briefly review the challenges that CI users face in their everyday lives. This will include problems arising due to room reverberation, ambient noise, and masking in general and the available signal processing solutions to alleviate these challenges.
3. Next Generation Signal Processing Steps for Cochlear Implants and Future Work: Here, we will summarize available signal processing techniques and the remaining obstacles of the field requiring more focused consideration.
Biographies
Oldooz Hazrati received her B.Sc., and M.Sc in Electrical Engineering from Tehran-Polytechnic University also known as AmirKabir University of Technology in 2005 and 2008, respectively. She received her Ph.D. in Electrical Engineering from The University of Texas at Dallas (UTD), Richardson, Texas, in 2012, under the supervision of Dr. Philip Loizou, with a research assistantship supported by grants from NIH and Cochlear Limited. In the spring of 2013, she started working as a post-doctoral research associate in the Cochlear Implant and Speech Processing laboratories in Erik Jonsson School of Engineering & Computer Science, The University of Texas at Dallas. Her research has been supported by a project from Cochlear Limited (PI: Oldooz Hazrati) since spring 2013 and a grant awarded from NIH. She has published 11 journal papers and 14 conference papers during her work at UTD on speech processing for Cochlear Implants.
John H.L. Hansen received his Ph.D. and M.S. degrees in Electrical Engineering from Georgia Institute of Technology, Atlanta, Georgia, in 1988 and 1983, and B.S.E.E. degree from Rutgers University, New Brunswick, N.J. in 1982. He joined University of Texas at Dallas (UTD), Erik Jonsson School of Engineering & Computer Science in 2005, where he is Associate Dean for Research and Professor of Electrical Engineering, and Professor in Brain and Behavioral Sciences (Speech & Hearing). At UTD, he holds the Distinguished Chair in Telecommunications Engineering, and established the Center for Robust Speech Systems (CRSS). From 1999–2005, he was with Univ. of Colorado Boulder, as Dept. Chair and Professor in Speech, Language, Hearing Sciences, and Professor in Electrical Engineering, and co-founded the Center for Spoken Language Research. From 1988–1998, he was with Duke Univ., Departments of Electrical and Biomedical Engineering, and founded the Robust Speech Processing Laboratory. He has served as IEEE Distinguished Lecturer, member of IEEE Signal Processing Society: Speech Technical Committee and Educational Technical Committee, Technical Advisor to U.S. Delegate for NATO (IST/TG-01), Associate Editor for IEEE Trans. Speech & Audio Proc., Associate Editor for IEEE Signal Proc. Letters, Editorial Board Member for IEEE Signal Proc. Magazine, and General Chair for Interspeech-2002, Organizer & Technical Chair for IEEE ICASSP-2010. He has supervised 68 thesis candidates, was recipient of the 2005 Univ. of Colorado Teacher Recognition Award, and author/co-author of 535 journal & conference papers in the field of speech processing and language technology, and co-author of Discrete-Time Processing of Speech Signals, IEEE Press, 2000.
Brett Swanson received a BE (Electrical) from the University of New South Wales in 1985, and a PhD from the University of Melbourne in 2008. He joined Cochlear Ltd in Sydney in 1992, and led the firmware development for the company’s first DSP- based sound processor. He was a key developer of the ACE and ADRO sound processing algorithms, which today are used by more than 100,000 cochlear implant recipients worldwide. His PhD thesis was entitled “Pitch perception with cochlear implants”. He has supervised two PhD students to successful completion. He regularly provides training to company employees, biomedical engineering students and visiting professionals.
Michael Goorevich received a BE (Electrical) from the University of Sydney in 1994, and a M.Sc. (Electronics) from Macquarie University, Sydney, in 2005. He joined Cochlear Ltd in Sydney in 1998, working as part of the team implementing signal and sound processing algorithms on DSP platforms. In 2003, he took on the role of team leader for the sound processing and firmware product development group in Sydney, and continued in that role until 2012. From 2012, he joined the research and technology group within Cochlear, co-leading a cluster of engineers and scientists working on research projects to develop new sound processing technologies for implantable hearing instruments.
T10—Compressive Covariance Sensing
Subject Area: Sparsity techniques
Presenters: Geert Leus (Delft University of Technology), Zhi Tian (George Mason University) and Daniel Romero (University of Vigo)
Less than 20 places left
Summary
There are many engineering applications that rely on frequency or angular spectrum sensing, such as cognitive radio, radio astronomy, radar, seismic acquisition and so on. Many of these applications do not require the reconstruction of the full signal and can perfectly rely on an estimate of the power spectral density (PSD) or, in other words, the second-order statistics of the signal. However, the large bandwidths of the involved signals lead to high sampling rates and thus high sampling costs, which can be prevented by a direct compression step carried out in the analog domain (e.g., by means of an analog-to-information converter, multi-coset sampling, analog beamforming, antenna selection, etc.). This leads to the problem of sensing the PSD or covariance using compressive observations, labeled as compressive covariance sensing (CCS). Another important application of this approach can be found in correlation mining, which is a technique to infer structure in large data networks. However, due to the magnitude of the networks, all the data cannot be directly exploited and some form of data-independent compression can be very helpful. CCS can be then used to perform correlation mining using these compressed signals.
In this tutorial, we will give an overview of the state of the art in CCS and present its connections to compressive sensing (CS). We focus on the design constraints of the compression matrices, which are completely different as in classical CS, and elaborate on the estimation/detection techniques to sense the covariance using compressive measurements. In this context, both non-uniform and random sampling are discussed. We further elaborate on distributed CCS, where compressive measurements in one domain are fused in the dual domain, i.e., temporal compressive measurements are gathered at different spatial sensors or spatial compressive measurements from different time slots are combined. Finally, connections to super-resolution techniques such as atomic norm minimization are discussed. We end this tutorial by sketching some open issues and presenting the concluding remarks.
Outline
1. Introduction
a. Spectrum Sensing
b. Compressive Spectrum Sensing
c. Applications
2. Compressive Covariance Sampling
a. Basic Principles
b. Covariance Structures
c. Estimation versus Detection
3. Covariance Estimation
a. Least Squares
b. Maximum Likelihood
c. Regularized Estimators
4. Covariance Detection
a. Neyman-Pearson
b. Multiple Hypothesis Testing
5. Non-Uniform Covariance Sampling
a. Linear Sparse Rulers
b. Circular Sparse Rulers
c. Dynamic Non-Uniform Sampling
d. Compression Limits
6. Random Covariance Sampling
a. Sampler Designs
b. Dynamic Random Sampling
c. Compression Limits
7. Distributed Covariance Sensing
a. Non-Uniform versus Random Sampling
b. Samples/Sensors Trade-Off
8. Super Resolution
a. Atomic Norm Minimization
b. Relation with Compressive Covariance Sensing
9. Open Issues
10. Conclusions
Biographies
Geert Leus (IEEE Fellow) received the electrical engineering degree and the PhD degree in applied sciences from the Katholieke Universiteit Leuven, Belgium, in June 1996 and May 2000, respectively. Currently, Geert Leus is an “Antoni van Leeuwenhoek” Full Professor at the Faculty of Electrical Engineering, Mathematics and Computer Science of the Delft University of Technology, The Netherlands. His research interests are in the area of signal processing for communications. Geert Leus received a 2002 IEEE Signal Processing Society Young Author Best Paper Award and a 2005 IEEE Signal Processing Society Best Paper Award. Geert Leus was the Chair of the IEEE Signal Processing for Communications and Networking Technical Committee, and an Associate Editor for the IEEE Transactions on Signal Processing, the IEEE Transactions on Wireless Communications, the IEEE Signal Processing Letters, and the EURASIP Journal on Advances in Signal Processing. Currently, he is a Member-at-Large to the Board of Governors of the IEEE Signal Processing Society and a member of the IEEE Sensor Array and Multichannel Technical Committee. He finally serves as the Editor in Chief of the EURASIP Journal on Advances in Signal Processing.
Zhi Tian (IEEE Fellow) received the B.E. degree from the University of Science and Technology of China, Hefei, China, in 1994, the M. S. and Ph.D. degrees from George Mason University, Fairfax, VA, in 1998 and 2000 respectively. From 2000 to 2014, she was a professor at Michigan Technological University. From 2011 to 2014, she was on leave from academia to serve as a Program Director at the US National Science Foundation (NSF). In January 2015, she joined the Electrical and Computer Engineering Department of George Mason University as a Professor. Her research interests lie in statistical signal processing, wireless communications and wireless sensor networks. She has been actively involved with the IEEE by serving regularly as a reviewer, associate editor, technical committee member, and conference technical program committee chair. She served as Associate Editor for IEEE Transactions on Wireless Communications and IEEE Transactions on Signal Processing. She is a Distinguished Lecturer of the IEEE Vehicular Technology Society from 2013 to 2015. Her program portfolios at NSF included communications, sensing and signal processing, and computation and data-enabled science and engineering.
Daniel Romero received the telecommunication engineering and the M.S. degrees from the University of Vigo (Spain) in 2009 and 2011, respectively, and will be defending his Ph.D. degree at the same university in the fall semester of 2014. He is currently a visiting scholar at the Signal Processing in Networking and Communications Group at the University of Minnesota, MN. In 2012, he visited the Circuits and Systems group at Delft University of Technology, The Netherlands. His main research areas are statistical signal processing, with a special interest in compressed sensing, compressed covariance sensing, estimation theory and detection theory; and machine learning, with applications in spectrum cartography and big data analytics.
T11—Over-The-Horizon Radar: Fundamental Principles, Adaptive Processing and Emerging Applications
Subject Area: Radar signal processing
Presenter: Giuseppe A. Fabrizio (Defence Science & Technology Organisation, Australia)
Summary
Skywave over-the-horizon (OTH) radars operate in the high frequency (HF) band (3–30 MHz) and exploit signal reflection from the ionosphere to detect and track targets at ranges of 1000 to 3000 km. The long-standing interest in OTH radar technology stems from its ability to provide persistent and cost-effective early-warning surveillance over vast geographical areas (millions of square kilometres). Australia is recognized as a world-leader in the OTH radar field. Pioneering research and development covering every facet of this technology has resulted in the multi-billion-dollar Jindalee Operational Radar Network (JORN) that consists of three state-of-the-art operational OTH radars in Australia.
The first part of the tutorial introduces the fundamental principles of OTH radar design and operation in the challenging HF environment to motivate and explain the architecture and capabilities of modern OTH radar systems. The second describes mathematical models characterizing the HF propagation channel and adaptive processing techniques for clutter and interference mitigation. The third delves into emerging applications, including HF passive radar, blind signal separation and multipath-driven geolocation. A highlight of the tutorial is the prolific inclusion of experimental results illustrating the application of robust signal processing techniques to real-world OTH radar systems. This is expected to benefit students, researchers and practitioners with limited prior knowledge of HF radar and with an interest in the application of advanced processing techniques to practical systems.
Outline
1. Fundamental OTH Radar Principles
a. Concept of operation and practical applications
b. System characteristics and nominal capabilities
c. Intelligent resource management and waveform design
d. Conventional processing for target detection and tracking
2. Advanced Adaptive Processing Techniques
a. Space-time model of HF propagation channel
b. Time-varying adaptive beamforming for nonstationary interference mitigation
c. Space-time adaptive processing (STAP) for interference and clutter mitigation
d. Adaptive CFAR detection based on the generalized likelihood ratio test (GLRT)
3. Emerging Research and Applications
a. HF passive radar (incl. experimental results)
b. Blind signal separation (incl. experimental results)
c. Multipath-driven geolocation (incl. experimental results)
d. MIMO radar concept (in brief if time permits)
Biography
Giuseppe A. Fabrizio received his B.E. and Ph.D. degrees from the Department of Electrical Engineering at Adelaide University, Australia, in 1992 and 2000. Since 1993, Dr Fabrizio has been with the Defence Science and Technology Organization (DSTO), Australia, where he leads the EW and adaptive signal processing section of the high frequency radar branch. Dr Fabrizio is responsible for the development and practical implementation of innovative and robust adaptive signal processing techniques to enhance the operational performance of the Jindalee Operational Radar Network (JORN). Dr Fabrizio is a senior member of the IEEE and is the principal author of over 50 peer-reviewed journal and conference publications. He is a co-recipient of the prestigious M. Barry Carlton Award for the best paper published in the IEEE Transactions on Aerospace and Electronic Systems (AES) on two occasions (2003 and 2004). In 2007, he received the coveted DSTO Science Excellence award for his contributions to adaptive signal processing in JORN. In the same year, he was granted a DSTO Defence Science Fellowship to pursue collaborative research at La Sapienza University in Rome, Italy. Dr Fabrizio has delivered OTH radar tutorials in the national and international IEEE Radar Conference series and is an Australian representative on the IEEE International Radar Systems Panel. He is currently serving as Vice President for Education on the AESS Board of Governors. Dr. Fabrizio was selected as the recipient of the distinguished IEEE Fred Nathanson Memorial Radar Award in 2011 for his contributions to OTH radar and radar signal processing. His textbook “High Frequency Over-the-Horizon Radar” was published by McGraw-Hill, NY, in June 2013.
T12—Convex Optimization for Big Data
Subject Area: Big data
Presenters: Volkan Cevher (EPFL), Mario Figueiredo (University of Lisbon), Mark Schmidt (University of British Columbia) and Quoc Tran-Dinh (EPFL)
Summary
This tutorial reviews recent advances in convex optimization algorithms for Big Data, which aim to reduce the computational, storage and communications bottlenecks. We provide an overview of this emerging field, describe contemporary approximation techniques, such as first-order methods and randomization for scalability and survey the important role of parallel and distributed computation. The new Big Data algorithms are based on surprisingly simple principles and attain staggering accelerations even on classical problems, some of which are well known in Signal Processing.
Outline
1. A Motivation: Convex optimization in the wake of Big Data.
2. First-order methods: Gradient-type, proximal-gradient-type, and primal-dual methods.
3. Randomization: Stochastic gradient methods, coordinate gradient methods, and randomized linear algebra.
Biographies
Volkan Cevher received the B.S. (valedictorian) degree in electrical engineering in 1999 from Bilkent University in Ankara, Turkey, and he received the Ph.D. degree in Electrical and Computer Engineering in 2005 from the Georgia Institute of Technology in Atlanta. He held research scientist positions at the University of Maryland, College Park from 2006 to 2007 and at Rice University in Houston, Texas, from 2008 to 2009. Currently, he is an Assistant Professor at the Swiss Federal Institute of Technology Lausanne with a complimentary appointment at the Electrical and Computer Engineering Department at Rice University. His research interests include signal processing theory, machine learning, graphical models and information theory. He received a Best Paper Award at SPARS in 2009 and an ERC StG in 2011.
Mário A. T. Figueiredo (IEEE Fellow) received MSc (1990), PhD (1994), and Habilitation (2004) degrees in electrical and computer engineering, all from Instituto Superior Técnico (IST), the engineering school of the former Technical University of Lisbon (now University of Lisbon). Since 1994, he has been with the Department of Electrical and Computer Engineering, IST, where he is now a Professor. He is also area coordinator and group leader at Instituto de Telecomunicações (a private non-profit research institute). His research interests include image processing and analysis, machine learning, and optimization. He is a Fellow of the IEEE and of the IAPR, and was the (co)recipient of several awards, namely the 2011 IEEE Signal Processing Society Best Paper Award, the 2014 IEEE W. R. G. Baker Award, and several best conference paper awards. His name is included in the Thomson Reuters’ Highly Cited Researchers list. He has been associate editor of several journals (namely, IEEE Transactions on Image Processing, IEEE Transactions on Pattern Analysis and Machine Intelligence, SIAM Journal on Imaging Sciences, Journal of Mathematical Imaging and Vision), and presented invited lectures and tutorials in many conferences and workshops.
Mark Schmidt is an assistant professor working in the field of machine learning and large-scale optimization in the Department of Computer Science at the University of British Columbia. He previously worked in the Natural Language Laboratory at Simon Fraser University and, from 2011 through 2013, worked at the École Normale Supérieure in Paris on inexact and stochastic convex optimization methods. He finished his M.Sc. in 2005 at the University of Alberta working as part of the Brain Tumour Analysis Project, and his Ph.D. in 2010 at the University of British Columbia working on graphical model structure learning with L1-regularization. He has also worked at Siemens Medical Solutions on heart motion abnormality detection, and with Michael Friedlander in the Scientific Computing Laboratory at the University of British Columbia on semi-stochastic optimization methods.
Quoc Tran Dinh received his B.S. degree in applied mathematics and informatics and the M.S. degree in computer science, both from Vietnam National University, Hanoi, Vietnam, in 2001 and 2004, respectively, and his Ph.D degree in electrical engineering from the Department of Electrical Engineering and Optimization Engineering Center (OPTEC), KU Leuven, Leuven, Belgium. He is currently a Postdoctoral Researcher with the Laboratory for Information and Inference System, École Polytechnique Federale de Lausanne, Lausanne, Switzerland. His research interests include theory and methods for convex optimization, sequential convex programming, parametric optimization and methods for variational inequalities and equilibrium problems.
T13—Adaptation, Learning, and Optimization over Networks
Subject Area: Network distributed signal processing
Presenter: Ali H. Sayed (University of California, Los Angeles)
Summary
There are many good reasons for the peaked interest in distributed implementations, especially in this day and age when the word “network” has become commonplace whether one is referring to social networks, power networks, transportation networks, data networks, biological networks or other types of networks. Some of these reasons have to do with the benefits of cooperation in terms of improved performance and improved resilience to failure. Other reasons deal with privacy and secrecy considerations where agents may not be comfortable sharing their data with remote fusion centers. In other situations, the data may already be available in dispersed locations, as happens with cloud computing. One may also be interested in learning through data mining from Big Data sets.
Motivated by these considerations, this tutorial deals with the topic of information processing over graphs and how collaboration among agents in a network can lead to superior adaptation and learning performance. The presentation covers results and tools that relate to the analysis and design of networks that are able to solve optimization, adaptation and learning problems in an online and distributed manner from streaming data through localized interactions among their agents. The results discussed in this tutorial are useful in comparing network topologies against each other and in comparing networked solutions against centralized or batch implementations. The results are also useful in elucidating how cooperation among unsophisticated agents can lead to powerful network behavior.
It is not difficult to envision that future engineering systems will benefit from similar bottom‐up design approaches involving coordination among less powerful units to achieve higher levels of cognition and performance. Robotic swarms are one notable example where agents can work together through an evolving topology, and adjust their locations and exploration space in response to environmental conditions, malfunctioning of neighbors or even suspicious behavior by intruders. Another example is the use of networked learners to mine information from Big Data sets, such as those related to health informatics, transportation networks, power grids, social networks, or surveillance applications. In these scenarios, it is often the case that information is already spread across dispersed locations and decentralized learning and adaptation offers an attractive approach to information processing.
The tutorial will discuss at length how families of networked agents can be made to adapt and learn continually from streaming data and from limited interactions among neighboring agents, and how by tapping into the “wisdom” of the group, networks are able to complement the limitations of their individual agents. The presentation will also explain how such networks can be used to solve optimization problems in a decentralized manner. The tutorial will further clarify the limits of performance of distributed solutions and discuss procedures that help bring forth their potential more fully. The presentation adopts a useful statistical framework and discusses performance results that elucidate the stability, convergence, and performance behavior of the learning networks. The work also illustrates how distributed processing over graphs gives rise to some revealing phenomena due to the coupling effect among agents. These phenomena are discussed in the context of several applications including distributed sensing, intrusion detection, dictionary learning, distributed estimation, distributed optimization, online learning, network system theory, biological networks and machine learning.
Outline
1. Motivation & Examples: Distributed processing. Biological networks. Adaptive networks. Graphs.
2. Optimization by Single Agents: Risks and loss functions. Convergence. Limitations.
3. Stochastic Optimization by Single Agents: Learning. Adaptation. Convergence. Limitations.
4. Performance of Single Agents: Analysis. MSE design. Logistic regression. Pattern classification.
5. Centralized Adaptation and Learning: Comparison with non‐cooperative schemes.
6. Multi-Agent Networks: Graph models and properties. Pareto optimality. Limits points.
7. Multi-Agent Distributed Strategies: Incremental, consensus, diffusion, primal-dual techniques.
8. Evolution of Multi-Agent Networks: Coupling among agents. Stochastic models. Implications.
9. Stability of Multi-Agent Networks: Bounded moments. Convergence. Cooperation matters.
10. Long-Term Network Dynamics: Slow adaptation regime. Network evolution.
11. Performance of Multi-Agent Networks: Performance expressions. Influence of topology.
12. Benefits of Cooperation: Doubly vs. left-stochastic policies. Social benefit vs. individual benefit.
13. Combination Policies: Optimal policies. Adaptive policies. Clustering.
14. Extensions: Gossip & asynchronous strategies, constrained optimization, sparse optimization.
15. Applications: distributed optimization, adaptation and learning, dictionary learning, biological networks, intrusion detection, target tracking, machine learning and online learning.
Biography
Ali H. Sayed (IEEE Fellow) is professor and former chairman of electrical engineering at the University of California, Los Angeles, where he directs the UCLA Adaptive Systems Laboratory. An author of over 440 scholarly publications and six books, his research involves several areas including adaptation and learning, statistical signal processing, distributed processing, network science and biologically-inspired designs. His work has been recognized with several awards including the 2014 Athanasios Papoulis Award from the European Association for Signal Processing, the 2013 Meritorious Service Award and the 2012 Technical Achievement Award from the IEEE Signal Processing Society, the 2005 Terman Award from the American Society for Engineering Education, the 2003 Kuwait Prize, and the 1996 IEEE Donald G. Fink Prize. He served as a 2005 Distinguished Lecturer for the IEEE Signal Processing Society. His articles received several Best Paper Awards from the IEEE Signal Processing Society in 2002, 2005, and 2012. He is a Fellow of both the IEEE and the American Association for the Advancement of Science (AAAS); the publisher of the journal Science. He is also recognized as a Highly Cited Researcher by Thomson Reuters.
T14— Imaging and Calibration for Aperture Array Radio Telescopes
Subject Area: Image processing
Presenters: Amir Leshem (Bar-Ilan University) and Stefan J. Wijnholds (Netherlands Institute for Radio Astronomy)
Summary
Calibration and imaging of a new generation of radio telescopes based on phased array technology poses a number of challenges: a low SNR per receiving element (typically < –20 dB), a large field-of-view (up to 4π sr) and direction dependent propagation conditions and instrument response. Dealing with these effects requires advanced algorithms that need to work through huge data volumes, which limits the acceptable numerical complexity. In this tutorial, we give an overview of the current state of the art in radio astronomical imaging and (self-) calibration and discuss the limitations of these methods. We then discuss a selected number of routes that are currently being explored to overcome these limitations. The tutorial will provide a complete overview of state-of-the-art imaging techniques for synthetic-aperture arrays and will give the attendees a clear view of the main challenges in future radio-astronomical imaging research.
Several new radio telescopes based on phased-array technology have recently become operational. The large field of view provided by this technology should improve the surveying and transient detection capabilities of these instruments by more than an order of magnitude compared to more conventional dish arrays, but this large field of view also poses new signal processing challenges, that should be dealt with to fully exploit these capabilities. The commissioning of these instruments has stimulated a burst of research on signal processing techniques for aperture-array radio telescopes as demonstrated by the considerable amount of recent signal processing papers in the technical as well as the astronomical literature. Significant progress in this area is still needed to attain the science potential of these instruments and mature phased-array technology for radio astronomy in the context of the Square Kilometre Array (SKA). The SKA is a future radio telescope, with receivers to be placed in South Africa and Australia, that is envisaged to be over an order of magnitude more sensitive than any telescope built to date and for which detailed designs are currently being made.
Outline
1. Introduction
a. Historical overview of instrument developments
b. Current scientific challenges
c. Imaging challenges
2. Signal processing formulation of calibration and imaging problems
a. Data model or measurement equation
b. Imaging theory and concepts
c. Calibration and imaging as parameter estimation problems
d. Extension to polarization measurements
3. Imaging
a. Fourier transform based imaging
i. Gridding + FFT
ii. Facet imaging
iii. W-projection
iv. A-projection
v. AW-projection
vi. Snapshot imaging
b. Model based imaging
i. Maximum-likelihood (weighted LS, KLT)
ii. Maximum-entropy method
iii. L1-based imaging
c. Dynamic range limitations of gridded imaging
d. High-dynamic-range imaging
4. (Self-) Calibration
a. Calibration scenarios
b. Direction independent calibration
i. Multi-source calibration
ii. Reduction of computational complexity by ADI methods
iii. Redundancy calibration
c. Direction dependent calibration
i. Weighted alternating least squares
ii. Space Alternn (SAGECal)
iii. Low-parametric models for direction dependent effects
d. Self-calibration: combining imaging and calibration
5. Current challenges and research directions
a. Deconvolution techniques
b. Improving computational efficiency
c. Dealing with strong sources
d. Source separation
e. Sparse reconstruction
f. Noise floor analysis
Biographies
Amir Leshem received the B.Sc.(cum laude) in mathematics and physics, the M.Sc. (cum laude) in mathematics, and the Ph.D. in mathematics all from the Hebrew University, Jerusalem, Israel, in 1986, 1990 and 1998 respectively. From 1998 to 2000, he was with Faculty of Information Technology and Systems, Delft university of technology, The Netherlands, as a postdoctoral fellow working on algorithms for the reduction of terrestrial electromagnetic interference in radio-astronomical radio-telescope antenna arrays and signal processing for communication. From 2000 to 2003, he was director of advanced technologies with Metalink Broadband where he was responsible for research and development of new DSL and wireless MIMO modem technologies and served as a member of ITU-T SG15, ETSI TM06, NIPP-NAI, IEEE 802.3 and 802.11. From 2000 to 2002, he was also a visiting researcher at Delft University of Technology working on imaging for radio astronomy.
He is one of the founders of the new school of electrical and computer engineering at Bar-Ilan university where he is currently a Professor and head of the Signal Processing track. From 2003 to 2005, he also was the technical manager of the U-BROAD consortium developing technologies to provide 100 Mbps and beyond over DSL lines. He was the leading guest editor of a special issue of IEEE Journal on Selected Topics in Signal Processing, dedicated to signal processing for space research and for a special issue of the Signal Processing Magazine, dedicated to signal processing in astronomy and cosmology. Since 2008, he is an associate editor for IEEE Trans. on Signal Processing.
Prof. Leshem has an ongoing ISF grant on signal processing for large synthesis aperture radio telescopes and is a partner on a Dutch science foundation program on signal processing for LOFAR.
His main research interests include multichannel wireless and wireline communication, applications of game theory to dynamic and adaptive spectrum management of communication and sensor networks, array and statistical signal processing with applications to multiple element sensor arrays and networks in radio-astronomy, brain research, wireless communications and radio-astronomical imaging, set theory, logic and foundations of mathematics.
Stefan Wijnholds received the M.Sc. degree (cum laude) in astronomy and the M.Eng. degree (cum laude) from the University of Groningen, The Netherlands, in 2003 and the Ph.D. degree (cum laude) from Delft University of Technology, Delft, The Netherlands, in 2010. After his graduation in 2003, he joined the R&D Department of the Netherlands Institute for Radio Astronomy (ASTRON), where he works with the System Design and Integration group on the development of the next generation of radio telescopes. He has developed calibration and imaging methods for the aperture array stations of the Low Frequency Array (LOFAR) and a calibration strategy for phased array feeds for dish telescopes. He also made important contributions to the array design of the LOFAR and is currently extending that work to the design of the Square Kilometre Array. From 2006 to 2010, he was also affiliated with the Delft University of Technology to pursue his Ph.D. on “Fish-Eye Observing with Phased Array Radio Telescopes”.
He received travel grants for the URSI GASS 2008 in Chicago (Ill.), the Asia-Pacific Radio Science Conference 2010 in Toyama, Japan, and the URSI GASS 2011 in Istanbul, Turkey. He is a partner on a Dutch science foundation program on signal processing for LOFAR.
His research interests lie in the area of array signal processing, specifically calibration of and imaging with sensor arrays. He also studies the fundamental limitations of such methods to establish design requirements for the next generation of radio telescopes.
T15—Perceptual Metrics for Image and Video Quality in a Broader Context: From Perceptual Transparency to Structural Equivalence
Subject Area: Image processing
Presenters: Sheila S. Hemami (Northeastern University) and Thrasyvoulos N. Pappas (Northwestern University)
Summary
We will examine objective criteria for the evaluation of image quality that are based on models of visual perception. Our primary emphasis will be on image fidelity, i.e., how close an image is to a given original or reference image, but we will broaden the scope of image fidelity to include structural equivalence. We will also discuss no-reference and limited-reference metrics. We will examine a variety of applications with special emphasis on image and video compression. We will examine near-threshold perceptual metrics, which explicitly account for human visual system (HVS) sensitivity to noise by estimating thresholds above which the distortion is just-noticeable, and supra-threshold metrics, which attempt to quantify visible distortions encountered in high compression applications or when there are losses due to channel conditions. We will also consider metrics for structural equivalence, whereby the original and the distorted image have visible differences but both look natural and are of equally high visual quality. We will also take a close look at procedures for evaluating the performance of quality metrics, including database design, models for generating realistic distortions for various applications and subjective procedures for metric development and testing. Throughout the course, we will discuss both the state of the art and directions for future research.
Outline
1. Applications: Image and video compression, restoration, retrieval, graphics, etc.
2. Human visual system review
3. Near-threshold perceptual quality metrics
4. Supra-threshold perceptual quality metrics
5. Structural similarity metrics
6. Perceptual metrics for texture analysis and compression – structural texture similarity metrics
7. No-reference and limited-reference metrics
8. Models for generating realistic distortions for different applications
9. Design of databases and subjective procedures for metric development and testing
10. Metric performance comparisons, selection, and general use and abuse
11. Embedded metric performance, e.g., for rate-distortion optimized compression or restoration
12. Metrics for specific distortions, e.g., blocking and blurring
13. Metrics for specific attributes, e.g., contrast, roughness, and glossiness
14. Multimodal applications
Biographies
Thrasyvoulos N. Pappas (IEEE Fellow) received the S.B., S.M., and Ph.D. degrees in electrical engineering and computer science from MIT in 1979, 1982, and 1987, respectively. From 1987 until 1999, he was a Member of the Technical Staff at Bell Laboratories, Murray Hill, NJ. He is currently a professor in the Department of Electrical and Computer Engineering at Northwestern University, which he joined in 1999. His research interests are in image and video quality and compression, image and video analysis, content-based retrieval, perceptual models for multimedia processing, model-based half-toning and tactile and multimodal interfaces. Prof. Pappas has served as co-chair of the 2005 SPIE/IS&T Electronic Imaging Symposium, and since 1997 he has been co-chair of the SPIE/IS&T Conference on Human Vision and Electronic Imaging. Dr. Pappas is a Fellow of IEEE and SPIE. He has also served as editor-in-chief of the IEEE Transactions on Image Processing (2010–12), elected member of the Board of Governors of the Signal Processing Society of IEEE (2004–06), chair of the IEEE Image and Multidimensional Signal Processing (now IVMSP) Technical Committee, technical program co-chair of ICIP-01 and ICIP-09, and co-chair of the 2011 IEEE IVMSP Workshop on Perception and Visual Analysis. He has also served on the editorial boards of the IEEE Transactions on Image Processing, the IEEE Signal Processing Magazine, the Journal of Electronic Imaging, and the Foundations and Trends in Signal Processing.
Sheila S. Hemami (IEEE Fellow) received the B.S.E.E. degree from the University of Michigan in 1990, and the M.S.E.E. and Ph.D. degrees from Stanford University in 1992 and 1994, respectively. She was with Hewlett-Packard Laboratories in Palo Alto, California in 1994 and was with the School of Electrical Engineering at Cornell University from 1995¬–2013. She is currently Professor and Chair of the Department of Electrical & Computer Engineering at Northeastern University in Boston, MA. Dr. Hemami’s research interests broadly concern communication of visual information from the perspectives of both signal processing and psychophysics. She was elected a Fellow of the IEEE in 2009 for her for contributions to robust and perceptual image and video communications. Dr. Hemami has held various visiting positions, most recently at the University of Nantes, France and at École Polytechnique Federale de Lausanne, Switzerland. She has received numerous university and national teaching awards, including Eta Kappa Nu’s C. Holmes MacDonald Award. She was a Distinguished Lecturer for the IEEE Signal Processing Society in 2010–11, was editor-in-chief for the IEEE Transactions on Multimedia from 2008-10. She has held various technical leadership positions in the IEEE.
T16—Beyond Randomness: Sparse Signal Processing in Practice
Subject Area: Sparsity techniques
Presenters: Waheed U. Bajwa (Rutgers University) and Marco F. Duarte (University of Massachusetts, Amherst)
Summary
While sparsity has been invoked in signal processing for more than four decades, it is only in the last decade or so that we have come to understand many of the theoretical underpinnings of sparse signal processing. These recent theoretical developments have given the practitioners in numerous application areas a number of insights. Many of these theoretical developments have come in the context of linear regression, statistical model selection and sampling/ill-posed inverse problems. These insights in turn have a potential to affect practice in the areas of biomarker identification from DNA microarray data, optical, hyperspectral and medical imaging, radar, sonar and array processing, sensor networks, wireless communications, etc. Potential impact of these developments is evidenced by the fact that, since 2006, ICASSP, ICIP, and SSP have collectively held close to a dozen tutorials that deal with various aspects of sparse signal processing, such as some of the basic theory with respect to sampling, quantization and optimization techniques, and its applications, such as MRI, optical imaging and medical imaging.
Despite the recent developments reported in these tutorials, practitioners in many areas can be found asking the following question: how can I either design reliable systems that adhere to the physical constraints posed by my application or guarantee that a given system or set of data are sufficient for reliable statistical inference? This is because initial literature on sparse signal processing and, in turn, earlier tutorials, have focused on randomized arguments and unstructured assumptions that give confidence to practitioners about the usefulness of sparse signal processing but fail to provide a verifiable theory of sparse signal processing. Consider, for instance, the widely referenced “restricted isometry property” (RIP) in the context of compressive sampling. While random measurement matrices have been shown to satisfy the RIP with high probability, very few practitioners are aware of how these results translate to arbitrary, often structured, measurement matrices arising in many real-world applications. This is because the RIP cannot be explicitly verified in polynomial time for a given measurement matrix. Similar limitations also exist for the theory of sparse signal processing in the context of linear regression, model selection, etc.
Our goal in this tutorial is to help practitioners in the transition from the theory of sparse signal processing to its real-world implementation, especially for those working in applications where the standard random designs are not translatable into a practical exploitation of signal sparsity. To achieve this goal, we will leverage some of our recent results, as well as several from other researchers in the area, and connect these results to applications in model selection, linear regression and sampling in the context of wireless communications, cellular networks, optical imaging, radar and analog-to-digital conversion applications. Because of the nature of the tutorial, we believe it will be of interest to a broad range of audience in the signal processing community who do not directly work in the sparse signal processing area but who are interested in figuring out quick means of translating theory to practice. We also believe that this tutorial will be of interest to a broad range of audience from industry, where sparse signal processing is still being evaluated in terms of its potential for changing the design of commercial systems.
Outline
1. Sparse Signal Processing: Signal Recovery/Reconstruction, Model Selection, and Regression
2. Computable metrics for arbitrary measurement/design matrices
3. Worst-case performance guarantees
4. Probabilistic signal models and average-case performance guarantees
5. Sparse Signal Processing in the Presence of Structured Sparsity
6. Connecting structured sparsity, group sparsity and multiple-measurement vector problems
7. Computable metrics for arbitrary measurement/design matrices in structured sparsity
8. Worst-case performance guarantees
9. Probabilistic signal models and average-case performance guarantees
10. Applications of Non-Random Sparse and Structured-Sparse Signal Processing
11. Wireless communications
12. Channel estimation in point-to-point communications
13. Activity recognition in cellular networks
14. Compressive imaging
15. Implementable structured measurement matrices
16. Deterministic constructions of sensing matrices
17. Radar target detection
18. Design of deterministic radar waveforms through the use of computable metrics
Biographies
Waheed U. Bajwa received BE (with Honors) degree in electrical engineering from the National University of Sciences and Technology, Pakistan, in 2001, and MS and PhD degrees in electrical engineering from the University of Wisconsin-Madison in 2005 and 2009, respectively. He was a Postdoctoral Research Associate in the Program in Applied and Computational Mathematics at Princeton University from 2009 to 2010, and a Research Scientist in the Department of Electrical and Computer Engineering at Duke University from 2010 to 2011. He is currently an Assistant Professor in the Department of Electrical and Computer Engineering at Rutgers University. His research interests include high-dimensional inference and inverse problems, sampling theory, statistical signal processing, computational harmonic analysis, machine learning, wireless communications and applications in biological sciences, complex networked systems and radar & image processing.
Dr. Bajwa has more than three years of industry experience, including a summer position at GE Global Research, Niskayuna, NY. He received the Best in Academics Gold Medal and President’s Gold Medal in Electrical Engineering from the National University of Sciences and Technology (NUST) in 2001, the Morgridge Distinguished Graduate Fellowship from the University of Wisconsin-Madison in 2003, and the Army Research Office Young Investigator Award in 2014. He co-guest edited a special issue of Elsevier Physical Communication Journal on “Compressive Sensing in Communications” (2012), co-organized 1st CPS Week Workshop on Signal Processing Advances in Sensor Networks (2013) and co-chaired IEEE GlobalSIP Symposium on New Sensing and Statistical Inference Methods (2013). He is currently an Associate Editor of the IEEE Signal Processing Letters and Publicity & Publications Chair of the 6th IEEE Intl. Workshop on Computational Advances in Multi-Sensor Adaptive Processing (2015).
Marco F. Duarte received the B.Sc. degree in computer engineering (with distinction) and the M.Sc. degree in electrical engineering from the University of Wisconsin-Madison in 2002 and 2004, respectively, and the Ph.D. degree in electrical engineering from Rice University, Houston, TX, in 2009.
He was an NSF/IPAM Mathematical Sciences Postdoctoral Research Fellow in the Program of Applied and Computational Mathematics at Princeton University, Princeton, NJ, from 2009 to 2010, and in the Department of Computer Science at Duke University, Durham, NC, from 2010 to 2011. He is currently an Assistant Professor in the Department of Electrical and Computer Engineering at the University of Massachusetts, Amherst, MA. His research interests include machine learning, compressed sensing, sensor networks and computational imaging.
Dr. Duarte received the Presidential Fellowship and the Texas Instruments Distinguished Fellowship in 2004 and the Hershel M. Rich Invention Award in 2007, all from Rice University. He coauthored (with C. Hegde and V. Cevher) the Best Student Paper at the 2009 International Workshop on Signal Processing with Adaptive Sparse Structured Representations (SPARS). He is also a member of Tau Beta Pi.
T17—Adaptive Learning for Model-Based Blind Source Separation
Subject Area: Audio signal processing
Presenter: Jen-Tzung Chien (National Chiao Tung University)
Less than 20 places left
Summary
This tutorial will introduce the state-of-art machine learning approaches for model-based blind source separation (BSS) with applications for speech recognition, speech separation, instrumental music separation, singing-voice separation and music information retrieval. The traditional BSS approaches based on independent component analysis were designed to resolve the mixing system by optimizing a contrast function or an independence measure. The underdetermined problem in the presence of more sources than sensors may not be carefully tackled. The contrast functions may not flexibly and honestly reflect the independence for an optimization with convergence assuming the static mixing condition could not catch the underlying dynamics in source signals and sensor networks. The uncertainty of system parameters may not be truly characterized so that the robustness against adverse environments is not guaranteed. The temporal structures in mixing systems as well as source signals may not be properly captured. The model complexity or the dictionary size we assume may not be fitted to the true one in source signals. With the remarkable advances in machine learning algorithms, the issues of underdetermined mixtures, optimization of contrast function, non-stationary mixing condition, multidimensional decomposition, ill-posed condition and model regularization have been resolved by introducing the solutions of nonnegative matrix factorization, information-theoretic learning, online learning, Gaussian process, sparse learning, dictionary learning, Bayesian inference, model selection, tensor decomposition and deep neural networks. This tutorial will present how these algorithms are connected and why they work for source separation particularly in speech and music applications. We start from the survey of BSS applications and model-based approaches. The fundamental theories including optimization algorithms, information theory, Bayesian learning, variational inference and Monte Carlo Markov chain inference will be addressed. A series of case studies are then introduced to deal with a variety of issues in model-based BSS. At last, we will point out a number of directions and outlooks for future studies.
Outline
1. Blind Source Separation (BSS)
a. Audio, speech and music processing
b. Independent component analysis (ICA)
c. Adaptive learning algorithms
d. Modern model-based approaches
2. Machine Learning Theories
a. Modeling and optimization
b. Information-theoretical learning
c. Nonnegative matrix factorization (NMF)
d. Sparse learning
e. Bayesian learning
f. Variational Bayesian inference
g. Monte Carlo Markov chain inference
3. Case Studies
a. Independent voices for adaptation
b. Nonparametric likelihood ratio ICA
c. Convex divergence ICA
d. Nonstationary Bayesian separation
e. Sequential Gaussian process for BSS
f. Sparse group learning for BSS
g. Bayesian selection and separation using NMF
h. Tensor decomposition and separation
i. Deep neural network for BSS
4. Summary and Future Trends
Biography
Jen-Tzung Chien received his Ph.D. degree in electrical engineering from National Tsing Hua University, Hsinchu, Taiwan, in 1997. During 1997–2012, he was with the National Cheng Kung University, Tainan, Taiwan. Since 2012, he has been with the Department of Electrical and Computer Engineering, National Chiao Tung University (NCTU), Hsinchu, where he is currently a Distinguished Professor. He serves as an adjunct professor in the Department of Computer Science, NCTU. He held the Visiting Researcher positions at the Panasonic Technologies Inc., the Tokyo Institute of Technology, the Georgia Institute of Technology, the Microsoft Research Asia and the IBM T. J. Watson Research Center. His research interests include machine learning, blind source separation, speech recognition, face recognition and information retrieval. He served as the associate editor of the IEEE Signal Processing Letters in 2008–2011, the guest editor of the IEEE Transactions on Audio, Speech and Language Processing in 2012, the organization committee member of the ICASSP 2009, and the area coordinator of the Interspeech 2012. He is appointed as the APSIPA Distinguished Lecturer for 2012-2013. He received the Distinguished Research Award from the Ministry of Science and Technology, Taiwan in 2006 and 2010. He was a co-recipient of the Best Paper Award of the IEEE Automatic Speech Recognition and Understanding Workshop in 2011. Dr. Chien currently serves as an elected member of the IEEE Machine Learning for Signal Processing Technical Committee and has served as the Tutorial Speaker for ICASSP 2012 at Kyoto, Interspeech 2013 at Lyon, APSIPA 2013 at Kaohsiung, and ISCSLP 2014 at Singapore.
