The fifth IEEE Workshop on 
Human-in-the-Loop Methods and Future of Work 
in BigData (HMData 2021) co-located with IEEE Bigdata 2021 (online)
Dec.15th (Planned)
!!! Deadline Extended !!!
Oct 15, 2021: Due date for workshop papers submission (all categories)
     (Authors have to submit the title and abstract by Oct. 8)


HMData workshop, which originally started as the "Human-Machine
collaboration in BigData" workshop, will investigate the 
opportunities and challenges in human machine collaboration in 
work with bigdata, which are described by two terms: Human-in-
the-Loop Methods and Future of Work. Human-in-the-Loop is a 
term focusing on the employer's viewpoint while Future of Work 
focuses more on worker's viewpoint, in both of which the division 
of labor among humans and machines is a key issue. This area is 
likely to be heavily AI driven, and we intend to invite papers 
covering the following aspects, (a) Capturing human capabilities
through intelligent models and how to adapt them through changing 
perceptions, needs, and skills. (2) High level tools that provide 
the ability for all stakeholders in the new ecosystem, including
regulators for policies and AI workers, to specify their 
requirements. (3) system design and engineering of job platforms 
for collection, storage, retrieval, and analysis of data deluge 
about workers, jobs, and their activities. (4) Benchmarking and 
the development of appropriate metrics to measure system 
performance as well as human aspects, such as satisfaction, 
capital advancement, and equity.

We welcome any interesting ideas and results on any relevant 
topics, but we also encourage submitting papers on new 
projects inspired by the COVID-19 crisis, such as those on 
human-in-the-loop solutions in the pandemic, those on re-evaluating 
how we organize labor and how we share work with machines in the 
future. To make the workshop an attractive place for those people, 
we solicit practitioner papers as well as research papers, in order 
to facilitate discussion among researchers who know solutions and 
practitioners who know problems. We also would like to make the 
place valuable for young researchers. All papers accepted for the 
workshop will be included in the Workshop Proceedings published by 
the IEEE Computer Society Press, made available at the Conference.


This workshop covers a wide range of topics of human-machine 
collaboration in work with bigdata. Keywords include: crowdsourcing, 
collaborative recommendation, crowdsensing, workflow model for 
humans and machines, incentives, human-assisted bigdata analysis, 
bigdata-human interaction, human-machine collaboration in real-world 
applications (such as natural disaster response, education, and citizen 
science), and ELSI in Human-in-the-loop systems and Future of Work. 

We expect submissions to address some of the following issues: 
- capturing human characteristics and capabilities,
- stakeholder requirement specification,
- social processes around the human-in-the-loop systems,
- platforms and ecosystems,
- computation capabilities, and
- benchmarks and metrics for human-in-the-loop systems and Future of Work


Lora Aroyo (Google)

Bio: Lora Aroyo is a Full Professor in Computer Science, currently 
working as a visiting research faculty at Google, NYC. Previously 
she was a visiting scholar at the Columbia Data Science Institute 
at Columbia University, New York. She is also Chief of Science for 
a NY-based startup Tagasauris, which works on hybrid machine learning 
and human-assisted computing strategies to enrich multimedia (e.g. video, 
images, and text) with meaningful information about its content, and 
ultimately improve video search and discovery. Lora is an active member 
of the Human Computation, User Modeling & Semantic Web communities. 
She is president of the User Modeling community UM Inc, which serves 
as a steering committee for the ACM Conference Series “User Modeling, 
Adaptation and Personalization” (UMAP) sponsored by SIGCHI and SIGWEB. 
She is also a member of the ACM SIGCHI conferences board. Since 2010 
she has actively worked towards shaping the concept of “User-Centric 
Data Science“, which ultimately led to the forming of and heading 
the User-centric Data Science group at the Department of Computer 
Science, Vrije Universiteit Amsterdam, The Netherlands. As an expert 
in user-centric data science, Lora conceived the vision of an 
user-centric experimental lab for computer science researchers at 
the VU University Amsterdam. She headed the team that made it possible 
in 2010 to open VU INTERTAIN Lab – the first of its kind in an academic 
environment.  Throughout her carrier, Lora was a principal investigator 
of a large number of research projects, she organized conferences, 
workshops, and tutorials to bring together methods and tools from 
human computation, linked (open) data, data science & human-computer 
interaction with the goal of building hybrid human-AI systems for 
augmenting both machine and human intelligence for understanding text, 
images, and videos with humans-in-the-loop and machines-in-the-loop. 
Her research projects focussing on semantic search, recommendation 
systems, personalized access to online multimedia collections have 
a major impact and established her as a recognized leader in human 
computation techniques for specific domains, such as digital humanities, 
cultural heritage, and interactive TV.
Important Dates 
- Oct 15 (Fri), 2021: Due date for workshop papers submission (Extended)
  (Authors have to submit the title and abstract by Oct. 8 (Fri))
- Nov 10 (Wed), 2021: Notification of paper acceptance to authors
- Nov 20 (Sat), 2021: Camera-ready of accepted papers
- Dec 15-18 (Wed-Sat), 2021: Workshops
All submissions must be submitted electronically through CyberChair.
Please prefix your submission category such as [Research Paper] to
the Title of Paper field in the submission page.
For example, if you would like to submit a project-in-progress paper
"Crowd-centric Approach to Digital Archive Maintenance," you have
to put "[project-in-progress paper] Crowd-centric Approach to Digital
Archive Maintenance" into the Title of Paper field.

All papers accepted for the workshop will be included in the Workshop 
Proceedings published by the IEEE Computer Society Press, made 
available at the Conference.
Submission Categories

Research Papers (*) (long presentation): They report significant and 
original results relevant to the scope of this workshop. We solicit 
innovative or thought-provoking work but they do not necessarily 
have to reach the level of completion. The expected length is between 
4 and 6 pages. The maximum length is 10 pages, though the paper 
should be commensurate with the size of the contribution.

Practitioner papers (*)(long presentation): They present interesting 
problems that require human-in-the-loop solutions in a variety of 
application domains, or present the interesting results of applying 
existing human-in-the-loop solutions to their domains. The expected 
length is between 4 and 6 pages. The maximum length is 10 pages, though 
the paper should be commensurate with the size of the contribution.

Project-in-progress papers (short presentation): They present the goals, 
challenges, and preliminary results of research or real-world projects 
in progress. The maximum length is 3 pages.

(*) Some of the papers submitted to the research or practitioner paper 
categories may be accepted as project-in-progress papers and allotted 
to short presentation slots.

Papers should be formatted to IEEE Computer Society Proceedings 
Manuscript Formatting Guidelines in the IEEE Bigdata 2021 CFP page

Senjuti Basu Roy (NJIT)
Alex Quinn (Pardue University)
Atsuyuki Morihsima (Univesity of Tsukuba)

Program Committee (To be extended)

Yukino Baba (University of Tsukuba)
Wolf-Tilo Balke (Technische Universitaet Braunschweig)
Ria Mae Borromeo (University of the Philippines Open University)
Francois Charoy (University of Lorraine, Inria, CNRS)
Marina Danilevsky (IBM Research - Almaden)
Ashraf Dewan (Curtin University)
Gianluca Demartini (University of Queensland)
Shady Elbassuoni (American University of Beirut)
Ujwal Gadiraju (Delft University of Technology)
David Gross Amblard (Rennes 1 University / IRISA Lab)
Munenari Inoguchi (University of Toyama)
Vana Kalogeraki (Athens University of Economics and Business)
Masaki Matsubara (University of Tsukuba)
Satoshi Oyama (Hokkaido University)
Raghav Rao (University of Texas at San Antonio)
Naoki Sakai (National Research Institute for Earth Science and Disaster Resilience)
Keishi Tajima (Kyoto University)
Hisashi Toriya (Akita University)
Saravanan Thirumuruganathan (QCRI)
Demetris Zeinalipour (University of Cyprus)