• Marcelino S. da Silva
  • Carlos R. L. Francês
  • João C. W. A. Costa
  • Diego L. Cardoso




Markov Decision Process, Quality of Service, Smart Grid


In Smart Grids, a variety of new applications are available to users of the electrical system (from consumers to the electric system operators and market operators). Some applications such as the SCADA systems, which control generators or substations, have consequences, for example, with a communication delay. The result of a failure to deliver a control message due to noncompliance of the time constraint can be catastrophic. On the other hand, applications such as smart metering of consumption have fewer restrictions. Since each type of application has different quality of service requirements (importance, delay, and amount of data to transmit) to transmit its messages, the policy to control and share the resources of the data communication network must consider them. In this paper Markov Decision Process Theory is employed to determine optimal policies to explore as much as possible the availability of throughput in order to transmit all kinds of messages, considering the quality of service requirements defined to each kind of message. First a non-preemptive model is formulated and after that a preemptive model is derived. Numerical results are used to compare FIFO, non-preemptive and preemptive policies.


[1] M. Sooriyabandara, J. Ekanayake. “Smart Grid – Technologies for its realization”, IEEE International Conference on
Sustainable Energy Technologies (ICSET), Sri Lanka, 2010.
[2] G. W. Arnold. “Challenges and Opportunities in Smart Grid: A Position Article”. Proc. of IEEE, vol. 99, no. 6, pp.
922–927, 2011.
[3] E. Ancillotti, R. Bruno, M. Conti, “The role of communication systems in smart grids: Architectures, technical solutions
and research challenges”. Comput. Commun., vol. 36, no 17-18, pp. 1665-1697, 2013.
[4] S. Latre, B. D. Vleeschauwer, W. V. Meerssche, K. D. Schepper, C. Hublet, W. V. Leekwijck, F. D. Turck. “PCN
Based Admission Control for Autonomic Video Quality Differentiation: Design and Evaluation”. J. Netw. Syst.
Manage., vol. 19, no. 1, pp. 32-57, 2011.
[5] R. M. Salles, J. M. A. Carvalho. “An Architecture for Network Congestion Control and Charging of Non-cooperative
Traffic”. J. Netw. Syst. Manage., vol. 19, no. 1, pp. 367-393, 2011
[6] P. L. Montessoro, “Efficient Management and Packets Forwarding for Multimedia Flows”. J. Netw. Syst. Manage, vol.
21, no. 2, pp. 194-217, 2013.
[7] S. Das, Y. Ohba, M. Kanda, D. Famolari, S. K. Das. “A key management framework for AMI networks in smart grid”.
IEEE Commun. Mag., vol. 50, no. 8, pp. 30-37, 2012.
[8] K. Tomsovic, D. E. Bakken, V. Venkatasubramanian, A. Bose. “Designing the Next Generation of Real-Time Control,
Communication, and Computations for Large Power Systems”. Proc. of the IEEE, vol. 93, no. 5, pp. 965-979, 2005.
[9] L. Du, Q. Liu. “The Design of Communication System on the Real-Time Relay Protection Based on GOOSE”. AsiaPacific Power and Energy Engineering Conference (APPEEC), China, 2012.
[10] V. Medina, I. Gomez, J. Luque, S. Martin. “ESTELLE: a method to analyze automatically the performance of
telecontrol protocols in SCADA systems”. IEEE Trans. Power Deliv. vol. 17, no. 3, pp. 712- 717, 2002.
[11] Y. Ting, Z. Zhidong, W. Jiaowen, L. Ang, “Research on Transmission Data System of Smart Grid based on IPv6
DiffServ Model”. Asia-Pacific Power and Energy Engineering Conference (APPEEC), China, 2010.
[12] M. Rahnamay-Naeini, Z. Wang, A. Mammoli, M. M. Hayat, “Impacts of control and communication system
vulnerabilities on power systems under contingencies”. IEEE Power and Energy Soc. Gen. Meet., USA, 2012
[13] T. Sauter, M. Laboshov, “End-to-end communication architecture for smart grid”. IEEE Trans. on Ind. Electron., vol.
58, no 4, pp. 1218- 1228, 2011.
[14] G. Bumiller, L. Lampe, H. Hrasnica, “Power Line Communication Networks for Large-Scale Control and Automation
Systems”. IEEE Commun. Mag., vol. 48, no. 4, pp. 106-113, 2010.
[15] C. Müller, H. Georg, C. Wietfeld, “A modularized and distributed simulation environment for scalability analysis of
smart grid ICT infrastructures”. International Conference on Simulation Tools and Techniques (SIMUTOOLS), Italy,
[16] D. E. Bakken, A. Bose, C. H. Hauser, D. E. Whitehead, G. C. Zweigle, “Smart Generation and Transmission With
Coherent, Real-Time Data”. Proc. of the IEEE, vol. 99, no. 6, pp. 928-951, 2011.
[17] Alcatel-Lucent Enterprise. Technology White Paper - Smart Choices for the Smart Grid. http://enterprise.alcatellucent.com/private/images/public/si/pdf_smartChoice.pdf. Accessed 03 February 2013.
[18] A. Vallejo, A. Zaballos, J. M. Selga, J. Dalmau, “Next-generation QoS control architectures for distribution smart grid
communication networks”. IEEE Commun. Mag. vol. 50, no. 5, pp. 128-134, 2012.
[19] M. Islam, H. Lee, “IEC61850 based operation, control and management of utility connected microgrid using wireless
technology”. International Conference on Intelligent Computing (ICIC), China, 2012.
[20] H. C. Tijms. “Stochastic models: an algorithmic approach”. John Wiley & Sons, Michigan, 1994.
[21] Y. Yan, Y. Qian, H. Sharif, David Tripper. “A survey on Smart Grid Communication Infrastructures: Motivations,
Requirements and Challenges”. IEEE Communications Surveys & Tutorials, vol 15, no 1, pp. 5–20, 2013.
[22] IEC 61850: Communication Networks and Systems in Substations. IEC standard, 2003.
[23] National Institute of Standards and Technology. NIST Special Publication 1108 – NIST Framework and Roadmap for
Smart Grid Interoperability Standards. Release 1.0, 2010.
[24] R. Jain, "The Art of Computer Systems Performance Analysis: Techniques for Experimental Design, Measurement,
Simulation, and Modeling," Wiley- Interscience, New York, NY, April 1991.
[25] O. Sigaud, O. Buffet. “Markov decision Process in Artificial Intelligence”. John Willey & Sons, USA, 2010.




How to Cite

Marcelino S. da Silva, Carlos R. L. Francês, João C. W. A. Costa, & Diego L. Cardoso. (2014). QOS MANAGEMENT IN SMART GRIDS: A MARKOVIAN APPROACH. Journal of Microwaves, Optoelectronics and Electromagnetic Applications (JMOe), 13(2), 122–138. https://doi.org/10.1590/2179-10742014v13i2392



Regular Papers