A Continuous Media Data Rendering System For

A Continuous Media Data
Rendering System For
Visualizing Psychological
Impression-Transition
†
‡
Fujiko Yara ([email protected])
Naofumi Yoshida ([email protected])
‡ Shiori Sasaki ([email protected])
† Yasushi Kiyoki ([email protected])
†
Faculty of Environmental Information, Keio University
‡ Graduate School of Media and Governance, Keio University
Application Example and
Demonstration
• We will show the demonstration of our
system.
• Our system makes a user’s feeling change
by rendering media data continuously.
I feel lonely,
tired….
Fufufu♪
Lighting
Picture
Smell
Music
Overview
In this presentation, we show
• Implementation method of continuous
media data rendering system to
visualizing a psychological impressiontransition.
• Experimental results for the feasibility of
our system.
Background Issues
Data Engineering Research Field
• Several media-data-search methods based on a
vector space models have been proposed.
ex. Music retrieval [IK_2005][IKNS_2003]
Image retrieval [KK_1994]
Psychology Knowledge
• Impression-Transition Models have been defied.
ex. Hevner-model [H_1937]
CIS (Color Image Scale) -model
[IK_2005] Ijichi, A. and Kiyoki, Y. : A Kansei Metadata Generation
Method for Interpretation, Information Modeling and Knowledge Bases, 16,
170-182, 2005.
[H_1937] Hevner, K : The affective value of pitch and tempo in music.
American Journal of Psychology, 49, 621-630, 1937.
[KK_1994]Kiyoki, Y. Kitagawa, T. and Hayama, T. : A metadatabase
system for semantic image search by a mathematical model of
meaning, ACM SIGMOD Record, Vol.23, No.4, 1994, 34-41.
[IKNS_2003] Ishibashi, N. Kiyoki, Y. Nakagami, Y. and Sato, A. : An
Impressionistic Metadata Extraction Method for Music Data with Multiple
Note Streams, DBSJ Letters, Vol.2, No.2, October 2003, pp.61-64 .
[KK_1993] Kitagawa, T. and Kiyoki, Y. :The mathematical model of
meaning and its application to multidatabase systems, Proceedings of 3rd
IEEE International Workshop on Research Issues on Data Engineering
Interoperability in Multidatabase Systems, April 1993, 130-135
System Overview
start-query
result 1
goal-query
・・・
Distance metric
Transition metric
Lw1
Lw2
■
goal▲
query
startquery
▲
Lw3■
Iw3▲
terminalmedia data
■
▲■
■
■
■▲
start-
▲Ｌw1
▲
■▲
▲
■
media data
■
■
CIS-model
▲Iw2
▲
■
■
Hevner-model
Impression-Transition DB
Vector Space
Start-point-query
Start-media data
Goal-point-query
Goal-media data
Color ＤＢ
Music ＤＢ
Sensor ＤＢ
Multi Media DB
Impression-Transition Model and
Derivation of Kansei
• The necessity of continuous impression-transition.
• In psychological impression-transition models,
media data are allocated continuously by
something to define.
• We think the continuous relationships of these
models show derivation of Kansei stay constant.
Allocated by
impression-words
Allocated by
color gradation
CIS-model
Hevner-model
Implementation Method (1/8)
• Step 1 : Creating an impression-words vector space
of media data and mapping the impression-words of
media contents into it.
1. We have implemented this system using
the Mathematical Model of Meaning
(MMM) [KK_1994].
2. The MMM search space is created by
using the Longman Dictionary of
Contemporary English.
3. The impression-words of media data are
mapped into this impression-word vector
space.
▲
▲
■
■
▲■
■
▲■
■
■
▲
▲
▲
■
▲
■
■
▲
■
▲
▲
■
■
■ Impression-Words in Longman Dictionary
▲ Impression-Words expressing Media data
•[KK_1994] Kiyoki, Y. Kitagawa, T. and Hayama, T. : A metadatabase system for semantic image search by a mathematical model
of meaning, ACM SIGMOD Record, Vol.23, No.4, 1994, 34-41.
•Longman Dictionary of Contemporary English, Longman, 1987
Implementation Method (2/8)
• Step 2 : Creating database representing the
route of the impression-transition.
1. By using psychological models as
impression-transition models, we
create databases which express the
route of the impression-transition.
2. Using the Hevner-model and the
CIS-model (Color Image Scale
model) as impression-transition.
Implementation Method (3/8)
• Step 3 : Submitting two query words (starting-query
and terminal-query) into the impression space.
1. A user submits two query words into
MMM search space created in step1.
2. Two query are not always words used
in Longman Dictionary, so in next step,
two query words are converted into
the words used in Longman
Dictionary.
terminal
-query▲
・
■
▲
terminal- ■
impression
start-▲ ■
▲ query
▲
■
■
・
▲
start-▲
■
impression
■
▲
■
■
■
▲
■
▲
■
■ Impression-Words in Longman Dictionary
▲ Impression-Words expressing Media data
● Query words
Implementation Method (4/8)
• Step 4 : Converting two query words as two
impression words (starting-impression and terminalimpression).
1. The starting-query is converted into
the semantically closest word
(starting-impression) within the
impression words included in the
route representing the impression
transition (Step2).
2. The terminal-query is also converted
into the semantically closest word
(terminal-impression).
terminal
-query▲
・
■
▲
terminal- ■
impression
■
▲
startingquery ▲
▲
■
■
・
▲
start-▲
■
impression
■
▲
■
■
■
▲
■
▲
■
■ Impression-Words in Longman Dictionary
▲ Impression-Words expressing Media data
● Query words
Implementation Method (5/8)
• Step 5 : Converting impression-words to media data
(starting-media data and terminal-media data)
terminalimpression ▲
startimpression
■
▲
■
■
▲ ■
■
▲
▲
terminalmedia data
▲
▲
■
start-media
data
■
■
▲
■
▲
■
■ Impression-Words in Longman Dictionary
▲ Impression-Words expressing Media data
Implementation Method (6/8)
• Step 6 :Extracting two media data from vector
space and mapping into the impression-transition
models respectively.
1. Two media data (Starting-media
data and terminal-media data) are
mapped into the database based on
the impression-transition models.
2. The appropriate route is chosen for
connecting starting-media data and
terminal-media data continuously.
Hevner-model
CIS-model
Implementation Method (7/8)
• Step 7 : Choosing the route on the impression-transition
model for rendering media data for visualization by
color data.
1. To realize rendering the media data
continuously, the route from startingmedia data to terminal-media data is
chosen.
2. According to taking time, the way how
to walk on the impression-transition is
decided.
Allocated by
color gradation
CIS-model
3. The way how to walk must be
uniformed like chess.
Allocated by
impression-words
Implementation Method (8/8)
• Step 8 ：Rendering the sequence of output media
data generated in Step7.
1. We render the set of color data
generated in Step7 along the selected
route using the impression-transition
model.
2. A personal computer is used to display
the rendering of the set of color data.
Experimental Results (1/3)
[confuse → comfortable]
Results
Example
• These results of the set of color data show the
feasibility of our continuous media data rendering
system for visualizing the change of psychological
impression-transition.
Hevner-model
CIS-model
(start-point-query=“confuse”, goal-point-query=”comfortable”
R=shortest (右回り))
(start-point-query=“confuse”, goal-point-query=”comfortable”
R=第４節5(B))
(start-point-query=“confuse”, goal-point-query=”comfortable”,
R=longest (左回り))
Experimental Results (2/3)
[merry → calm]
Results
Example
• These experiments have shown the applicability of our
method for user’s various requirement of impressiontransition.
CIS-model
Hevner-model
(start-point-query=“merry”, goal-point-query=“calm”
R=longest (Right and Down))
(start-point-query=“merry”, goal-point-query=“calm”
R=longest (Down and Right))
(start-point-query=”merry”, goal-point-query=”calm”
R=shortest (右回り))
(start-point-query=”merry”, goal-point-query=”calm”
R=longest (左回り))
(start-point-query=“merry”, goal-point-query=“calm”
R=shortest (Left and Down))
(start-point-query=“merry”, goal-point-query=“calm”
R=shortest (Down and Left))
Experimental Results (3/3)
[simple → graceful]
Results
Example
• Our method has applicability for various strength of relationship
between two query words, even if the two impression-words have a
weak relationship in the vector space.
CIS-model
Hevner-model
(start-point-query=“simple” goal-point-query=“graceful”,
R=longest (Right and Down))
(start-point-query=“simple” goal-point-query=“graceful”,
R=longest (Right and Down))
(start-point-query=“simple”,
goal-point-query=“graceful”)
(start-point-query=“simple” goal-point-query=“graceful”,
R=longest (Right and Down))
(start-point-query=“simple” goal-point-query=“graceful”,
R=longest (Right and Down))
Summary
• Our method makes it possible to implement
visualizations of the continuous change of
impression-transition, according to impressionwords expressed for starting point and terminal
point.
• By the implementation and experiments using
color data as output media data, we have
clarified the feasibility of our method for
visualizing the change of impression-transition
from starting point to terminal point by using the
research results of musical psychology and color
psychology.
Future Work
• We will design aggregate functions for
color data expression in the experiments
using psychological word groups by the
Hevner-model.
• We will approach to the computation
mechanisms of continuous transition of
impression.
References
•
•
•
•
•
•
•
•
[KK_1993] Kitagawa, T. and Kiyoki, Y. :The mathematical model of meaning and its
application to multidatabase systems, Proceedings of 3rd IEEE International Workshop on
Research Issues on Data Engineering Interoperability in Multidatabase Systems, April
1993, 130-135
[KK_1994] Kiyoki, Y. Kitagawa, T. and Hayama, T. : A metadatabase system for semantic
image search by a mathematical model of meaning, ACM SIGMOD Record, Vol.23, No.4,
1994, 34-41.
[KKH_1995] Kiyoki, Y. Kitagawa, T. and Hitomi, Y. : A fundamental framework for realizing
semantic interoperability in a multidatabase environment, Journal of Integrated ComputerAided Engineering, Vol.2, No.1, Jan.1995, 3-20.
[AS_1994] Aiello, R. and Slobada, J.A.: Musical perceptions, Oxford University Press,
1994.
[IK_2005] Ijichi, A. and Kiyoki, Y.:A Kansei Metadata Generation Method for Interpretation,
Information Modeling and Knowledge Bases, 16, 170-182, 2005.
Longman Dictionary of Contemporary English, Longman, 1987
[H_1937] Hevner, K. : The affective value of pitch and tempo in music, American Journal of
psychology, 49, 621-630.
[IKNS_2003] shibashi, N. Kiyoki, Y. Nakagami, Y. and Sato, A. : An Impressionistic
Metadata Extraction Method for Music Data with Multiple Note Streams, DBSJ Letters,
Vol.2, No.2, October 2003, pp.61-64 .
Thank you for your attention.
Derivation of Kansei
• 印象遷移モデルは一定を約束している
• 別のストーリを持つモデルは多く存在し、
• 微分係数が一定でないものが多い。
The Impression-Transition Model
and Liner relationship
• There is no relationship between color and impression-words.
20
１
２
３
19
４
conspicuous
bright
15
１０
• There is no relationship between color and impression-words.
Allocated by
impression-words
Hevner-model
Allocated by
color gradation
CIS-model
言葉を連続にした、
照明色ＤＢの構築
• 仮定１：心理情況遷移の仮定は連続し、
かつサイクルをなす。
• 仮定２：色相サイクルと、心理情況遷移
サイクルは対応する。
• 用語の設定
言葉：心理情況を示す単語
仮説の検証
マンセルの色円を用いた場合
20
１
２
３
19
４
15
１０
• 色円の代表色に番号を
つける。
• 言葉と色との間に線形
性があるかどうかを、
千々岩先生の色別イ
メージ・プロフィール(色
の印象評価アンケート
結果)を用い検討する。
上記の体表色は、マンセルにより20等分されたものである。
マンセルの色円と言葉の間に線形
性はあるか？
・・・・ないっぽ
い
25
20
15
軽快な
明るい
-5
10
5
19
15
18 17 19 17
14
11
5
3
0
-5
-10
-15
1 2 3 4 6 9 11 14 15 16 18 19
-10
10
派手な
目立つ
明るい
25
20
19 19 19
19
18
18
16
16
15
15
10
5
5
3
0
-5 1 2 3 4 6 9 11 14 15 16 18 19
-10
-15
-15
-20
1 2 3 4 6 9 11 14 15 16 18 -2
19
親しみ易い
-10
-15
18
9
軽快な
10
8
6
4
2
0
-2
-4
-6
-8
-10
1
15
5
-10
25
20
3
-5
2
-1
1 2 3 4 6 9 11 14 15 16 18 19 -4
1
11
目立つ
8
5
6
8 8
7
5
4
3
3
親しみ易い
-2
18
0
0
13
7
5
12
15
13
9
11
10
13
17
14
5
18
15
19
10
6
20
18 17 19
3
• 結果
派手な
-8

Download Report