The world has changed with the development of electronic systems and educational systems also use the products of the virtual world. E-learning enables teachers to efficiently cooperate with students in different scientific issues; it connects people all over the world and helps the fast, flexible and efficient implementation of the new scientific and educational materials. The National University of Public Service (NUPS) established its E-Learning Centre in 2013, which became a modern education centre for innovative public service. To further develop and internationalize the system, it is beneficial to find good practices from other higher education institutions. In this paper we shortly introduce the e-learning system of the NUPS and the e-learning system of University Fernando Pessoa (UFP) (Oporto, Portugal) called Virtual University. We exchange views and experiences by sharing our practices in the hope of implementing common development projects in the future.
Keywords: training for civil servants, e-learning courses, Pro Bono, UFP-UV, virtual room, webinar, Sakai Program
For many years in Hungary the constant further-training of practicing public administration officials was not ensured centrally, it happened in an isolated way. However, the efficiency of public administration has become more important than ever. That is why in 2012 the Hungarian Government appointed the National University of Public Service (NUPS) as the institution responsible for the continuous training and retraining of the whole public administration staff. NUPS, as a centre of excellence, keeps developing special training programs and provides further training for thousands of public servants each year. NUPS,
- 133/134 -
with the help of e-learning, provides high quality and effective training for civil servants at an affordable cost.
To develop and internationalize the Hungarian system, it is indispensable and highly beneficial to find good practices from other higher education institutions of the world. In this study, after shortly introducing the NUPS's e-learning system, we familiarize the readers with the e-learning system of University Fernando Pessoa (UFP) (Oporto, Portugal), called Virtual University.
The world has changed with the development of electronic systems and educational institutions make use of the products of the virtual world. At a time when change is faster than ever, a key advantage of e-learning is that it has quicker delivery cycle times than traditional classroom-based instructions.[1] E-learning is the new way to teach and to efficiently cooperate with students in different scientific issues. It can connect people all over the world and help the fast, flexible and efficient implementation of the new scientific and educational materials. In fact, research indicates that e-learning reduces learning time by at least 25 to 60 percent when compared to traditional learning.[2]
Analysing the Hungary situation, we can ascertain that this teaching method has become widespread in higher education. This innovation means that the modernization of higher education reflects the constant development of the world. E-learning has countless advantages compared to face-to-face learning methods. It can get to an unlimited number of users; that is why it is suitable for training numerous people. Advancement and efficiency are more important today for public administration than ever. Hence, the Hungarian Government decided to introduce e-learning to ensure further training of civil servants to save money, time, and energy.
In Hungary, especially in the public service, e-learning is a relatively recent phenomenon; however, the foundations have been laid down and its continuous improvement is under way. The National University of Public Service (NUPS) established its E-Learning Centre in 2013 and became a modern education centre for innovative public service. Unique and special training material is developed based on the needs of government staff. The university is using this teaching method for the training of practicing civil servants and sometimes for BA, MA courses; which means at least 100.000 civil servants and students per year participate in the e-learning programme. This is an easy and comfortable way for practising officials to improve their skills and knowledge online. The extensive circle of users requires modern e-learning solutions to suit their needs. Since practising public officials have
- 134/135 -
limited time beside their day to day work, the program owner has to elaborate time-saving solutions. The pedagogical objective of the e-learning method is to educate learners in the most effective and learner-centred way with clear understanding of the various topics and with the help of clear and user-friendly presentation of the teaching material, self-evaluation tests, providing additional features in order to intensify the learning experience.
The organisation establishing the Centre was the Institute of the Executive Training and Continuing Education (ETCE) responsible for organising the training, education and exams of all civil servants in Hungary, using the latest technology and training methodology: e-learning and blended learning. The main tasks of the ETCE are: to support career advancement in public service, to provide exclusive advanced training and examination system for Hungarian public administration, to do special trainings for public service, to re-train senior (retired) military and police officers to public administration and to operate new training systems for municipalities. In 2015 the growing demand for improvement received a new organisational background with the foundation Apertus Nonprofit Firm supported by the NUPS. This firm ensures further improvement and hosts the IT background for the successful education in public service.
The electronic training system for civil servants is called Pro Bono (probono.uni-nke.hu/nyitolap), which is a unique platform for organising and completing programs, trainings (face-to-face), e-learning and blended learning. The homepage is available only in Hungarian at the moment, simply because only Hungarian native speakers use it currently.
The operation of Pro Bono is divided between the participants: first, the training officers who are the human resource leaders of over 1800 institutions. Secondly, the training organizers who are responsible for organizing courses and exams on behalf of the institutions in charge (NUPS and program owners), and last but not least, the third group is the participants of the trainings, i.e. the civil servants. Civil servants are instructed to use the platform and to plan their own training programs on their own private page.
Currently there are more than 1800 training programs at Pro Bono.[3] The great majority of these programs are created and managed by the public service institutions. These are the "internal programs" which meet the necessity of partaking special expertise for the institutions. But there is a considerable amount of programs provided solely by NUPS; namely 227 public service training programs and 33 leader training programs. The number of these programs is growing steadily thanks to the cooperation between the institutions and NUPS and to the EU funds.
NUPS is committed to share Hungarian experiences on further training of public and civil servants. Therefore it offers short-cycle training programmes in various fields, serving government needs on a large scale.[4] Some of the main topics are: legislation, decision-making process of the government, management of central public administration, functions and procedures of central governance, public finances, budgeting, supervisory systems in public sector, e-public services, economy and development policies, labour policies, social policies, EU policies, IT security, cyber defence, communication, PR, media, HR management, local
- 135/136 -
governance, health care policies, etc.[5] But there are many other important programs, covering any institutional demand.
The benefits of online learning are plentiful. In this study we do not provide a detailed description of Pro Bono's operation; we merely give some revealing and insightful information and data on the matter. Pro Bono e-learning gives users the opportunity to create a fun and engaging training course through videos, interactive slides, and even games (sometimes humorous animated videos). The courses lead to better results by helping users retain more of the information learned. The online training system reduces costs that would normally be associated with classroom training, including travel, learning materials, venue, and catering. With Pro Bono the users can participate in their online courses at any time from any location - including from home, during down time at work, or anywhere else they feel appropriate. E-learning materials are stored online, so civil servants can access important resources any time they encounter a question or difficult situation.
There is a help desk for participants where they can download their electronically signed certificates, file a complaint report and fill in satisfaction questionnaires. These latest two are really useful functions, since they help developers get information on the efficiency of the program plus the problem areas identified by the complaints get addressed in the further development of the system. Pro Bono served 1863 institutions and 70481 public servants in 2015.[6] The system operates with the participation of 300 teachers and trainers; there are 500-1500 users per day on Pro Bono.
E-learning development is still a work in progress; the next step is the incorporation of NUPS BA, MA and PhD course videos. It needs personalization to suit the bigger variety of programs to personal competences, and it needs additional portal support. E-learning means no boundaries. Users are able to participate in online courses from anywhere with an internet connection. This eliminates a lot of energy spent coordinating where and when a course will take place. And most importantly users are able to get real-time feedback during an online training course. Additionally, because everything is connected online, organizers can also get up-to-date analysis on how courses are going, and track the progress of individual employees. That is why the main aim for the future is to create a face to face, interactive teaching methodology, to form and establish a system of virtual rooms. These rooms would serve as interactive forums or chat rooms to share knowledge, experience and scientific results online with the participation of people located anywhere in the country or the world.
To develop the system established in Hungary, we have to find good practices in other foreign higher education institutions. In September 2015 we had the opportunity to visit the University Fernando Pessoa (UFP) in Porto, Portugal. Our host professor Prof. Pedro Reis,
- 136/137 -
associate professor, head of the e-Learning Department UFP-UV (Virtual University) shared his views and experiences on the topic. He shared with us their good practices in the form of an interview. At the end of the interview we highlight shared points for future development, which can substantiate the future scientific and research activities of the universities. These activities can form the basis for the universities' future cooperation.
University Fernando Pessoa is funded and managed by Fernando Pessoa Foundation, which is its legal representative. UFP was recognized as a public interest organization by the Portuguese State in 1996 and is organised in three faculties (Health Sciences, Human and Social Sciences, Science and Technology) and one School of autonomous Health Sciences unit (Ponte de Lima College). The University is committed to contribute to the shaping of a truly inclusive European Area of Higher Education. Being a foundational University, UFP has never been market oriented. We are a solid private higher education institution with a strong academic focus and a constant concern for quality maintenance and improvement in what concerns a highly qualified teaching staff, physical infrastructures and equipments and its teaching, research and extension projects.
After approximately 30 years of on-site courses, in 2012, the decision was made at UFP to also offer distance courses.
We offer undergraduate, master and PhD courses. These are mostly from the Faculty of Human and Social Sciences, including undergraduate courses such as Communication Sciences, Business Sciences, Business Management and Accounting, Criminology, Political Science and International Relations, Psychology, Social Work, and master courses such as Creativity and Innovation; Humanitarian Action, Cooperation and Development; Clinical and Health Psychology; Legal Psychology, Teaching and Management of Education, and PhD in Information Sciences.
There are also postgraduate courses (for instance, Facial Expression of Emotion) and training courses (for instance Business English - elementary and intermediate).
Moreover, there are also blended-learning courses from different academic fields; e.g. Health Sciences (Nursing) and Technology (Computer Sciences).
Since 2012 about 300 national and international students have attended e-learning courses at UFP, in Portuguese, English, or Spanish.
This initiative represents a major contribution to the internationalization of the university, which is, at the current stage, among the most important of our strategic goals.
The implementation of Distance Education courses at UFP implied the creation of a new department, called UFP-UV (Universidade Virtual, that is Virtual University); it is a small department where I am in charge as Pedagogical Coordinator, there are two other permanent full-time collaborators (one for user support and content management and another one for technical support and development), and also some occasional trainees providing specific services to about 300 students and 80 faculty members - so far involved in this project.
- 137/138 -
The e-learning courses at UFP-UV use a Learning Management System (LMS) called Sakai. Developed by educators for educators, Sakai is a digital learning/teaching and academic collaboration environment that we customised to meet our needs and image.
The Sakai community is comprised of individuals and institutions that share a common commitment to enhancing teaching, learning and research. This commitment is reflected in how community members come together to pool knowledge, define the needs of academic users, create and share software tools, and support the adoption of the Sakai project. Participation from more than 350 of the world's leading institutions lends creativity, stability and longevity to this effort.[7]
In 2009, the UFP-UV team won the award Sakai fellow of the year, for the contributions made to the community of users, particularly with the creation of a tool, Site Stats, to see statistics and create field activity reports.
At the beginning of the implementation process, the aim was to give it an operational face, in order to make this teaching and learning process, new at the university, as comprehensible as possible for everybody to get involved (faculty, students and also some administrative staff). We were well aware of how challenging it was to adopt an approach which would be able to cover a significant number of people, since people would be heterogeneous, at the level of IT skills, for example. Additionally, this proved to be true not only for students but also for teachers, who, belonging to different generations and having different scientific backgrounds, naturally have very heterogeneous levels of computer skills.
Great emphasis has been placed on support technologies, and pedagogical models have been inspired by international best practices, adapted to our reality and carefully transmitted and discussed with teachers. Thus, continuous training and especially permanent support were and still are crucial to the project. Training and support to teachers are provided mostly F2F[8] at the university e-learning department and support to distance students is provided in various forms: e-mail, phone, Skype and also original tutorials produced at the department and sent by email or delivered on the LMS.
The training program for faculty members focuses on methodological, pedagogical and technological aspects. One of the biggest challenges, at first, was indeed the fact that teachers were not prepared for e-learning. This required very personalized training, in small groups, within the same scientific field to better realize and share the difficulties that e-learning could represent to each group. This is complemented by constant assistance from a help desk devoted exclusively to this function.
- 138/139 -
In the implementation of our e-learning comprehensive methodology, we had to insist with faculty members that courses would take place exclusively on the Virtual Campus of the university, which is equipped with appropriate state of the art educational technologies suitable for this purpose. This guarantees that students have online courses; furthermore, it is a guarantee for both teachers and the university, since their work is documented, and it can be demonstrated for national and international educational authorities how our e-learning courses work and what results have been achieved.
Other ways of communicating with students, by email or Skype, for example, are not prohibited, of course, but those are not to be confused with the official teaching-learning process. The latter occurs exclusively on the Virtual Campus of the university, whereas other contacts by email or Skype have to be considered as informal communication, as the equivalent of a corridor or cafeteria contact in an on-site context. Thus, in addition to focusing on the specific training program applying new technological educational tools, the training of faculty members involved a kind of evangelization to help bring about a teaching paradigm change.
Although the open source LMS forms the basis of the virtual campus, other open source educational technologies were added in an integrated approach. Thus, students do not have to leave the UFP-UV Campus to attend their courses. Here, they find all the resources used in the teaching-learning process: different tools of the LMS, especially for written asynchronous activities; the Virtual Room, for synchronous oral contacts; videos of the recorded lessons from their teachers; and the Learning Objects (LO), each having a main pedagogical goal, depending on the chosen theme of the distance courses (mentioned at the beginning).
The recorded lessons are produced with Educ@st, a video management system for recorded lessons of faculty, which can simultaneously capture the speaker and their PowerPoint or Prezi presentations. The equipment is installed in the TV studio of the university where specialised technicians and our content manager assist teachers in the production of videos, which typically last around 15 minutes, mostly focusing on theoretical issues. These videos are then uploaded to a central server, edited and published in the LMS (Sakai), so that there is a controlled access to students. Access can be done by streaming or by downloading to PCs and mobile devices for later use, since the final publication is available in three distinct formats (Flash, Quicktime, iPod), which allows students to access their courses anytime, anywhere.
The Virtual Room is called COLIBRI, a Web Collaborative Environment. With the COLIBRI service, users can have video conference sessions for multiple purposes (lessons, webinars - sometimes with invited lecturers from other universities, tutorial sessions, dissertation supervision, and meetings among students for group work). It includes features such as live quizzes, screen or file sharing and mobile device access. It is also cross browser compatible.
The Virtual Room is also integrated with LMS (Sakai). Students can attend lessons and webminars live or see them later, since they are all recorded. Therefore students with professional commitments, who cannot attend the live lessons or webinars, have the option to see them later and thus learn what was discussed. Those who participated can also review the
- 139/140 -
lessons as a study strategy. The synchronous sessions in the Virtual Room provide moments for clarification, reflection, discussion or synthesis of subject matters.
Some LO were produced using freeware authoring tools, such as eXe Learning, Course-Lab, and especially Xerte (a fully-featured e-learning development environment for creating web content, with enriched media and rich interactivity). With licensed software, more sophisticated LO could eventually be produced, but we had to consider both budget restraints and the long learning curve necessary to achieve proficiency in a new software which is hard to conciliate with the rapid development of e-learning. This led us to choose, for the moment, easy-to-use, yet relatively powerful, e-learning authoring tools which offer programming-free WYSIWYG environment for creating interactive e-learning content which can be published in our LMS. In spite of this effort, we recognize that the creation and development of LO is one of the fields where our activity could improve, as I will mention later.
The implementation of e-learning on a comprehensive scale has been progressive and in the future we are trying to develop more courses in purely e-learning formats, i.e., including recorded lessons, learning objects (multimedia and interactive modules), and other formative self-study materials (such as online tests).
We are well aware that not all scientific areas and not all education levels adjust equally to the development of multimedia and interactive LO. Furthermore, not all teachers have the same ability to interact with Instructional Designers in order to convert their working materials to e-learning formats.
However, with the adoption of an e-learning platform to complement F2F classes in 2004, the good practices of some teachers began to be known and appreciated by the community; students started asking teachers not yet using the platform to provide e-learning materials, schedule activities, allow online assignment submission, etc. In this case the former good examples excited students and pushed other teachers to adopt those good practices. We hope for a similar outcome in the case of e-learning courses. If certain courses provide students with attractive teaching materials, it creates a certain level of expectation that students will want to experience in other courses as well.
So we plan to encourage the use of diversified learning materials to make the courses more appealing, taking into account the adequacy of those materials to the nature of subject matters and the learning outcomes of each level (1[st], 2[nd] or 3[rd] cycle).
Another cornerstone in an e-learning system is the assessment scheme. We are well aware that the online assessment system is not infallible, but, is there any system that can be considered as such? However, our efforts are focused on looking for solutions that offer more guarantees to ensure the credibility of the courses, which we consider a key factor to uphold. At present, written exams are conducted online, via Virtual Room, using application sharing, with remote supervision of the teacher. The audio and video channels of both the student and the teacher must be open. Actually, the use of webcam by the student is mandato-
- 140/141 -
ry; otherwise the exam cannot take place. In order to further improve these guarantees, we consider introducing changes in the future, such as adding software that helps to ensure the identification of students in the written tests; or eventually establishing partnerships with foreign higher education institutions or protocols with official entities (embassies, consulates) where international students can go to take written tests and exams.
Generally speaking, tertiary education is engaged in a strong process of internationalisation: global markets of educational services are developing, for example. An increasing number of institutions in non-English speaking countries now offer courses in English to overcome their linguistic disadvantage in attracting foreign students.
Nowadays, internationalisation is definitely a key strategic goal at UFP, which is seen as a mechanism that can be used for profiling institutions, increasing the recruitment of students and for developing partnerships, strategic alliances and networks with foreign institutions, in addition to being a means of improving the quality of education and research.
This strategy includes, among other things, the development of e-learning study programs in foreign languages, namely English, the establishment of joint degrees offered in collaboration with foreign partners, the development of international research cooperation and the planned use of EU programs.
After closely examining the e-learning systems of NUPS and UFP we can identify several common points and challenges in the situation of both universities. These common points can be the basis for joint future research activities and development cooperation.
Both universities created stable and high quality systems, which perfectly suit the requirements of modern e-learning. At the beginning of the implementation processes, the aim was to give the system an operational face, in order to make this teaching and learning process as comprehensible as possible to everybody involved (faculty, students/civil servants and also some administrative staff). The users and the teachers are heterogeneous; the level of their IT skills, for example. That is why support technologies, and pedagogical models should be carefully conveyed and discussed with teachers. The implementation of e-learning systems on a comprehensive scale has been progressive and in the future both universities are trying to develop more courses in pure e-learning formats, i.e., including recorded lessons, learning objects (multimedia and interactive modules), and other formative self-study materials (such as online tests).
Certainly, there are some differences between the two systems apart from the IT solutions. One of the main differences is in the scope of recipients. UFP has developed distance education mainly for students - many of them living abroad - whereas the NUPS provides e-learning programs for practicing Hungarian civil servants, basically in Hungarian.
- 141/142 -
UFP offers courses in Portuguese, English or Spanish. We can observe another difference in the number of users. At UFP about 300 national and international students are attending e-learning courses, while at NUPS the Pro Bono served more than 1863 public administration institutions and 70481 public servants in 2015. But these are just the numbers; the two universities are facing similar challenges in the future from many points of view.
Designing an interactive and fun course through the use of multimedia or the more recently developed methods of gamification enhances the engagement factor and the relative lifetime of the course material. Studies have shown that gamification enhances learner engagement and improves retention. Thanks to the dozens of sophisticated learning tools available today, it is simpler to introduce gamification into a learning program.[9] We think that using more picturesque, gripping programs would make the learning process much more efficient.
UFP-UV has already introduced the virtual room, which, from our point of view, can even raise the number of graduates per year. With e-learning, it is possible to gain instant access to users and professors/trainers from all parts of the world. Besides enabling teams from various geographies to collaborate on problem-solving challenges in real-time, these tools also bring with them other advantages like instant connectivity to subject matter experts; regardless of where they are located.[10] It would create a win-win situation for the Hungarian system (users and institutions) too. Another potential in the virtual room is that the lecturer has the ability to host a guest lecture without having to spend much money. It can be done virtually, with cameras. The added benefit comes in when we are able to replay the lecture and gain even more out of it. Students that missed out can view the recording, or students that attended can watch it again to further their understanding.[11]
As we have already mentioned before, one of the common challenges is the teacher's ability to use this tool. Not all teachers have the same ability to interact online, or with the Instructional Designers, in order to convert their working materials to e-learning formats. The program managers should take care of preparing the teachers and should ensure a nonstop helpdesk.
The assessment scheme is also an important keystone. UFP-UV is making efforts to find solutions that offer more guarantees to ensure the credibility of the courses. It would be an important question for the Hungarian system as well, if the virtual room opens.
Another common point is the implementation of more courses fully completed for pure e-learning formats, i.e., including recorded lessons, learning objects (multimedia and interactive modules), and other formative self-study materials (such as online tests).
Nowadays, internationalisation is definitely a key strategic goal for both institutions. Our discussions opened a new opportunity for cooperation to improve the missing details together to make both e-learning systems more efficient. It could be beneficial to cooperate and carry out projects together on international level.
- 142/143 -
In order to provide distance learning programs, we believe that institutions should comply with quality standards, such as:
- have a clearly defined and stated mission, goals, and objectives;
- have reasonably attainable and clearly stated educational objectives, and educationally sound and up-to-date curricula that are supported by quality instructional materials and appropriate technology;
- provide satisfactory educational services;
- offer adequate student support services;
- have demonstrated ample student success and satisfaction through an outcomes assessment plan;
- have qualified faculty and competent administrators and staff;
- have fair admission policies and adequate enrolment agreements;
- advertise its courses/programs truthfully;
- be financially able to deliver high quality educational services;
- have fair and equitable tuition and refund policies;
- have adequate facilities, equipment, supplies, and record protection;
- conduct continuous research and self-improvement studies.[12] The development of on-line platforms facilitates the cooperation between higher-institutions and helps the improvement of our teaching methods by implementing the achievements of the digital world.
Both universities are highly student-centred. We look forward to future cooperation and to carrying out joint research projects in order to make our systems more efficient and accessible for students and public officials alike.
- 143/144 -
Lilla Mária Juhász (Juhasz.Lilla@vtki.uni-nke.hu) is the vice-director for project development at the National University of Public Service, Institute of Executive Training and Continuing Education, Hungary. She is also a doctoral candidate; her research fields are the educational systems of public administration, the ethics and integrity policy of public officials. She is the president of honour at the Association of Hungarian PhD and DLA Students (DOSZ), Public Administration Department. During her academic studies, she established the Hungarian community of PhD students in public administration science. In 2011, she took part in the establishment of the National University of Public Service (NUPS), she was the advisor and coordinator of the ministerial commissioner. She was the leader of the Centre for Hungarian State Reform Operative Programs at NUPS and managed ten EU programs during the period of 2013-2015. In 2015, she established the operation of the Institution of the Science of the State at NUPS. Currently she is planning the new developments of the public sciences and the Hungarian education system for public officials in the framework of 2015-2020 EU Development Strategy.
Csilla Paksi-Petró (csilla.petro@vtki.uni-nke.hu) is an assistant professor at National University of Public Service, Faculty of Public Administration, Institute of Career and Human Resources. Her research area is the public service carrier management, including further training and leadership training. She has five years practice in education/training: leadership skills, organisational development, communication, motivation and performance management, carrier management, time management, self-knowledge, conflict resolution, stress management, team-building and labour law. She also has five years practice in HRM as a practicing pubic administration expert: recruitment, interviews, questionnaires, assessment centre, development centre, further training, allowance system, job analyses and evaluation systems, outplacement. She was the manager of many public administration development projects financed by the EU such as recruitment program for one-stop shop officials (2013); job analysis project in the public service (2012-2013); Integral Evaluation System in the public service (2012-2013); leadership recruitment project in the public administration (2014).
Prof. Pedro Reis (preis@ufp.edu.pt) is Associate Professor at the Faculty of Human and Social Sciences at Fernando Pessoa University (Porto, Portugal) and Head of the e-Learning Department UFP-UV (Virtual University). He is co-founder of Centro de Estudos sobre Texto Informático e Ciberliteratura (Centre for Computer-produced Texts and Cyberliterature Studies), and researcher of several financed projects. He is co-director of the journal Cibertextualidades (Edições UFP). He developed a Post-Doctoral Research Project on e-Learning with the cooperation of the Instructional Technology and Distance Education (ITDE), Nova Southeastern University (NSU) (Florida, USA). He is currently collaborating with the IAEA (UN, Vienna) as Consultant on e-learning and Educational Technologist. His main research
- 144/145 -
interests are Experimental Literature; Cyberculture and Cyberliterature; Digital Humanities; E-learning; Educational Technology; Instructional Design, and e-Content Development. He published several books and articles and he lectured in Portugal, Sweden, Ireland, USA, France, Brazil, Spain, Holland, Italy, Senegal, Germany, Ghana and Austria. ■
NOTES
[1] www.kineo.com/us/resources/new-to-elearning/the-benefits-of-elearning (13/12/15)
[2] Brandon Hall: Learning management and Knowledge Management. Is the holy grail of integration close at hand?, 2001, www.brandonhall.com (13/12/15) and www.kineo.com/us/resources/new-to-elearning/the-benefits-of-elearning (13/12/15)
[3] www.apertus.hu/szolgaltatasaink (27/12/15)
[4] en.uni-nke.hu/study/training-for-civil-servants (23/12/15)
[5] Academic Report, National University of Public Service, 2014, 83.
[6] Datas of the National University of Public Service, ETCE, 2015.
[7] sakaiproject.org/node/43 (09/09/2015)
[8] F2F is IT slang meaning "Face-To-Face".
[9] elearningindustry.com/science-benefits-gamification-elearning (22/12/15)
[10] www.talentlms.com/elearning/benefits-and-drawbacks-of-online-learning (22/12/15)
[11] www.talentlms.com/elearning/benefits-and-drawbacks-of-online-learning (20/01/2016)
[12] www.deac.org/UploadedDocuments/2015%20Handbook/1-Pages-1-28-2015.pdf p. 6. (20/01/2016)
Visszaugrás
