Skip to main content
Thammasat University Digital Collections
Home
Browse All
Log in

Help

English
English
Deutsch
Español
Pirate English
한국어 Korean
Français
ไทย Thai
Search
Advanced Search
Find results with:
error div
Add another field
Search by date
Search by date:
from
after
before
on
from:
to
to:
Searching collections:
Ebook Collection
Add or remove collections
Home
Ebook Collection
Selected topics on continuoustime controlled Markov chains and Markov games
Reference URL
Share
Add tags
Comment
Rate
To link to this object, paste this link in email, IM or document
To embed this object, paste this HTML in website
Selected topics on continuoustime controlled Markov chains and Markov games
Download
small (250x250 max)
medium (500x500 max)
Large
Extra Large
large ( > 500x500)
Full Resolution
Print
There is no file associated with this item.
Description
Rating
Title
Selected
topics
on
continuoustime
controlled
Markov
chains
and
Markov
games
Creator
PrietoRumeau, Tomas.
Contributors
HernandezLerma, O. (Onesimo)
World Scientific (Firm)
DescriptionAbstract
This
book
concerns
continuoustime
controlled
Markov
chains
, also
known
as
continuoustime
Markov
decision
processes
. They
form
a
class
of
stochastic
control
problems
in
which
a
single
decisionmaker
wishes
to
optimize
a
given
objective
function
. This
book
is
also
concerned
with
Markov
games
,
where
two
decisionmakers
(or
players)
try
to
optimize
their
own
objective
function
.
Both
decisionmaking
processes
appear
in a
large
number
of
applications
in
economics
,
operations
research
,
engineering
, and
computer
science
,
among
other
areas
. An
extensive
,
selfcontained
,
uptodate
analysis
of
basic
optimality
criteria
(such
as
discounted
and
average
reward)
, and
advanced
optimality
criteria
(e.g.
,
bias
,
overtaking
,
sensitive
discount
, and
Blackwell
optimality)
is
presented
. A
particular
emphasis
is
made
on the
application
of the
results
herein
:
algorithmic
and
computational
issues
are
discussed
, and
applications
to
population
models
and
epidemic
processes
are
shown
. This
book
is
addressed
to
students
and
researchers
in the
fields
of
stochastic
control
and
stochastic
games
.
Moreover
,
it
could
be of
interest
also to
undergraduate
and
beginning
graduate
students
because
the
reader
is
not
supposed
to have a
high
mathematical
background
: a
working
knowledge
of
calculus
,
linear
algebra
,
probability
, and
continuoustime
Markov
chains
should
suffice
to
understand
the
contents
of the
book
.
DescriptionTable Of Contents
1
.
Introduction
.
1.1
.
Preliminary
examples
.
1.2
.
Overview
of the
book
.
1.3
.
Contents
.
1.4
.
Notation

2
.
Controlled
Markov
chains
.
2.1
.
Introduction
.
2.2
. The
control
model
.
2.3
.
Existence
of
controlled
Markov
chains
.
2.4
.
Exponential
ergodicity
.
2.5
.
Proof
of
Theorem
2.11
.
2.6
.
Conclusions

3
.
Basic
optimality
criteria
.
3.1
.
Introduction
.
3.2
. The
finite
horizon
case
.
3.3
. The
infinite
horizon
discounted
reward
.
3.4
. The
longrun
expected
average
reward
.
3.5
. The
vanishing
discount
approach
to
average
optimality
.
3.6
.
Pathwise
average
optimality
.
3.7
.
Canonical
triplets
and
finite
horizon
control
problems
.
3.8
.
Conclusions

4
.
Policy
iteration
and
approximation
theorems
.
4.1
.
Introduction
.
4.2
. The
policy
iteration
algorithm
.
4.3
.
Approximating
discounted
reward
CMCs
.
4.4
.
Approximating
average
reward
CMCs
.
4.5
.
Conclusions

5
.
Overtaking
,
bias
, and
variance
optimality
.
5.1
.
Introduction
.
5.2
.
Bias
and
overtaking
optimality
.
5.3
.
Variance
minimization
.
5.4
.
Comparison
of
variance
and
overtaking
optimality
.
5.5
.
Conclusions

6
.
Sensitive
discount
optimality
.
6.1
.
Introduction
.
6.2
. The
Laurent
series
expansion
.
6.3
. The
vanishing
discount
approach
(revisited)
.
6.4
. The
average
reward
optimality
equations
.
6.5
.
Strong
discount
optimality
.
6.6
.
Sensitive
discount
optimality
in the
class
of
stationary
policies
.
6.7
.
Conclusions

7
.
Blackwell
optimality
.
7.1
.
Introduction
.
7.2
.
Blackwell
optimality
in the
class
of
stationary
policies
.
7.3
.
Blackwell
optimality
in the
class
of
all
policies
.
7.4
.
Conclusions

8
.
Constrained
controlled
Markov
chains
.
8.1
.
Introduction
.
8.2
.
Discounted
reward
constrained
CMCs
.
8.3
.
Average
reward
constrained
CMCs
.
8.4
.
Pathwise
constrained
CMCs
.
8.5
. The
vanishing
discount
approach
to
constrained
CMCs
.
8.6
.
Conclusions

9
.
Applications
.
9.1
.
Introduction
.
9.2
.
Controlled
queueing
systems
.
9.3
. A
controlled
birthanddeath
process
.
9.4
. A
population
system
with
catastrophes
.
9.5
.
Controlled
epidemic
processes
.
9.6
.
Conclusions

10
.
Zerosum
Markov
games
.
10.1
.
Introduction
.
10.2
. The
zerosum
Markov
game
model
.
10.3
.
Discount
optimality
.
10.4
.
Average
optimality
.
10.5
. The
family
of
average
optimal
strategies
.
10.6
.
Conclusions

11
.
Bias
and
overtaking
equilibria
for
Markov
games
.
11.1
.
Introduction
.
11.2
.
Bias
optimality
.
11.3
.
Overtaking
optimality
.
11.4
. A
counterexample
on
bias
and
overtaking
optimality
.
11.5
.
Conclusions
.
Publisher
Imperial College Press
Distributed by World Scientific Pub. Co.
Subject
Markov processes.
Stochastic control theory.
Identifier (Full text)
9781848168497
(electronic
bk.)
;
1848168489
;
9781848168480
;
http://www.worldscientific.com/worldscibooks/10.1142/P829#t=toc
Language
eng
Type
Electronic books.
FormatExtent
xi, 279 p. : ill. (some col.)
Date
c2012
.
RelationIs Part Of
ICP advanced texts in mathematics
v. 5
OCLC number
874497358
CONTENTdm number
237
Tags
Add tags
for Selected topics on continuoustime controlled Markov chains and Markov games
View as list

View as tag cloud

report abuse
Comments
Post a Comment
for
Selected topics on continuoustime controlled Markov chains and Markov games
Your rating was saved.
you wish to report:
Your comment:
Your Name:
...
Back to top
Select the collections to add or remove from your search
A
B
C
D
E
F
G
H
I
J
K
L
M
N
O
P
Q
R
S
T
U
V
W
X
Y
Z
Select All Collections
E
Ebook Collection
EReference
ETDA Publications
I
ISEAS (Institute of Southeast Asian Studies)
S
Somdet Phra Nyanasamvara
Special project (Bachelor of Arts Program in Journalism and Mass Communication)
T
Thailand Research Fund (TRF)
Thammasat History Collection
Thammasat University Research
Thammasat University Textbooks
Thammasat University Theses
The 2011 Flood at TU
The Foundation for the Promotion of Social Sciences and Humanities
The Thai Democratization Center
W
Wat Bowonniwet Vihara Cremation Collection
500
You have selected:
1
OK
Cancel