Peter Jacko peter.jacko [a] gmail.com

Senior Lecturer (Associate Professor) in Management Science Department of Management Science, Lancaster University Management School, LA1 4YX, UK Senior Decision Scientist, Berry Consultants, Abingdon, UK At Lancaster University, I am also member of: Foundations theme, Data Science Institute research groups: Health Systems, Simulation and Stochastic Modelling and Optimisation STOR-i: Statistics and Operational Research Centre for Doctoral Training in Partnership with Industry

:: Research Overview

Peter Jacko is interested in the science of efficient management of complex business processes, systems, and networks. Much of his research work benefits from the interaction of mathematics, computer science, and economics. His publications contribute to areas such as operational research, performance evaluation, stochastic modelling, queueing theory, applied probability, and machine learning. These subjects provide foundation for the modern fields of business analytics and data science. His research is motivated by real-world problems in business decision-making, communications networks, and healthcare management. The leading theme of his research activities is modelling and designing tractable and well-performing strategies for efficient use of scarce resources in dynamic and stochastic environments.

His research interests are in:

• Fields: Operational Research / Management Science / Business Analytics, Statistics / Data Science / Machine Learning
• Areas: Reinforcement Learning, Stochastic Modelling, Performance Evaluation, Decision Theory, Multi-Stage Optimisation, Simulation, Queueing Theory, Applied Probability
• Problems: Resource Allocation, Scheduling, Optimisation and Learning under Uncertainty/Risk, Sequential Experiments, Multi-armed Bandits, Networks Optimisation
• Methods: Markov Decision Processes, (Approximate) Dynamic Programming, Stochastic/Bayesian Analysis, Monte Carlo Simulation, Heuristics

His research efforts have been motivated by and the results are aimed to apply to:

• Public Health: Adaptive Clinical Trials, Personalised Medicine
• Business: Retail, Contact Centres
• Communications Networks: Wireless Data Networks (D2D, 4G LTE), Internet (TCP, ICN)

See Google Scholar for the list of publications and co-authors.

:: Research Metrics

H-index: 15                   (source: Google Scholar, 2021)
Erdős number: 3        (source: MathSciNet Calculator, 2021)

:: Research Lines

WARNING: The text below hasn't been updated since 2013. See the Group on Optimal Adaptive Learning (G.O.A.L.) for my current research lines.

1. Wireless Networks / Scheduling under Time-Varying Service Rate
2. Call Centers / Scheduling under Abandonment
3. Dynamic and Stochastic Resource Capacity Allocation Problems / Restless Bandits
4. Internet Congestion Control / Time-Varying Input Stream
5. Revenue Management / Knapsack Problem for Perishable Items
6. Admission Control and Routing / Delayed Information
7. Finance / Risk Management
8. Railways / Minimal Energy Consumption
9. Frequency Assignment / Distance Colorings
10. Incentives / Decision Making

< Wireless Networks / Scheduling under Time-Varying Service Rate

< Call Centers / Scheduling under Abandonment

< Dynamic and Stochastic Resource Capacity Allocation Problems / Restless Bandits

Jacko (2009) unifies into a comprehensive framework several well-known problems in combinatorial optimization, management, stochastic scheduling, and queueing theory. The following paragraphs are dedicated to outline a couple of special cases of the stochastic and dynamic resource allocation problem known as the bandit problems.

Suppose that there are several competitors (or projects, or bandits) evolving in a stochastic manner, which simultaneously demand or service from a server that is able to serve only one competitor at time. We can decide to which competitor the server is allocated at any given moment. The competitors evolution yields certain aggregate reward, i.e., a sum of individual customer rewards that only depend on the current state of the competitor and whether it is currently being served or not. The objective is to allocate the server in such a way that the aggregate reward is maximal, either in discounted expectation or on time-average.

In a more general variant, we may consider a capacity of a resource divisible into a finite number of units and state-dependent (discrete) capacity demands from the competitors, instead of the server able to serve strictly one competitor at time. Since this problem is PSPACE-hard to solve optimally, we are typically interested in an approximate solution which is easy to implement in practice. However, for some special cases it is possible to characterize an optimal solution.

Note that the problem when the resource and the competitor demands (seen as goods and services) are continuously-divisible is the fundamental one in neoclassical economics build up from marginalism. It should therefore not be surprising that marginal concepts also appear in the (approximate) solution to the discretely-divisible problem considered here.

Multi-Armed Bandit Problem (Classic Bandits)

Imagine a gambler in casino faced with a (fixed) number of one-armed bandit machines (or slot machines). Suppose that she is there alone and forever, and every minute she can and must choose one of the machines to play (to pull the arm). The played machine randomly evolves and yields a payoff according to its current state. The objective of the gambler is to maximize the expected discounted payoff obtained over an infinite horizon.

Imagine a physician faced with a flow of patients with a new disease. Suppose that every day she can and must choose one of the available treatments to apply to the patient coming that day. The treated patient reacts to the treatment and it is immediately known whether the treatment was successful (healing the patient) or not (dying). The objective of the physician is to heal as many patients as possible, i.e., to maximize the expected discounted number of healed patients over an infinite horizon.

The above two real-life problems lead to the same mathematical model of learning known as the multi-armed bandit problem. Its main distinguishing feature is the freezing property: the states of non-played machines (or non-used treatments) do not change. In fact, in the first case, the gambler is to learn which machine is the best one on average, and to exploit all the machines in favorable states during the process of learning. Similarly in the second case, the physician is to learn which treatment is the best on average, and to exploit all the other treatments in favorable states during the process of learning.

Gittins and his colleagues in the early 1970s showed that there are certain state-dependent prices (now called the Gittins index values) that can be calculated for each bandit independently of the others, and such that the problem can be optimally solved by an index rule. The index rule prescribes to index (or sort) at every time period the bandits according to their prices (associated to their current states) and to play anyone of those with the highest price (colored in green in the video). The video shows evolution of the prices (or the Gittins index values) in the multi-armed bandit problem. Recall the freezing property: the prices of the non-played bandits do not change, since the states of such bandits do not change.

Restless Bandits

In the problem of restless bandits the freezing property is dropped. This model was introduced by Whittle (1988), where he proposed to calculate the price (sometimes called the Whittle index value) that generalizes the Gittins index value to this case. However, the Whittle index values do not necessarily exists, and further, the index rule is in general non-optimal. Nevertheless, a growing body of literature shows that for many real-life problems the existence of such prices can be established and that the index rule performs, on average, very close to optimality.

< Internet Congestion Control / Time-Varying Input Stream

< Revenue Management / Knapsack Problem for Perishable Items

< Admission Control and Routing / Delayed Information

< Finance / Risk Management

< Railways / Minimal Energy Consumption

< Frequency Assignment / Distance Colorings

< Incentives / Decision Making

Last modified: 11/2018