PCEtLXtzdWJ0ZW1wbGF0ZSBjb21tb24vaGVhZGVyX2NvbW1vbn0tLT4JPCEtLXtpZiAkX0dFVFsnbW9kJ10gPT0gJ3ZpZXd0aHJlYWQnICYmICFlbXB0eSgkX0dbJ3RpZCddKX0tLT4KCTxsaW5rIHJlbD0iY2Fub25pY2FsIiBocmVmPSJodHRwczovL3d3dy5xaWRhbzEyMy5jb20vYmJzL3RocmVhZC17JF9HWyd0aWQnXX0tMS0xLmh0bWwiIC8+Cgk8IS0te2Vsc2VpZiAkX0dFVFsnbW9kJ10gPT0gJ2ZvcnVtZGlzcGxheScgJiYgIWVtcHR5KCRfR1snZmlkJ10pfS0tPgoJPGxpbmsgcmVsPSJjYW5vbmljYWwiIGhyZWY9Imh0dHBzOi8vd3d3LnFpZGFvMTIzLmNvbS9iYnMvZm9ydW0teyRfR1snZmlkJ119LTEuaHRtbCIgLz4KCTwhLS17ZWxzZWlmICRfR0VUWydtb2QnXSA9PSAnZ3VpZGUnfS0tPgoJPGxpbmsgcmVsPSJjYW5vbmljYWwiIGhyZWY9Imh0dHBzOi8vd3d3LnFpZGFvMTIzLmNvbS9iYnMvZ3VpZGUvIiAvPgoJPCEtLXtlbHNlaWYgJF9HWydiYXNlc2NyaXB0J10gPT0gJ2ZvcnVtJyAmJiAoZW1wdHkoJF9HRVRbJ21vZCddKSB8fCAkX0dFVFsnbW9kJ10gPT0gJ2luZGV4Jyl9LS0+Cgk8bGluayByZWw9ImNhbm9uaWNhbCIgaHJlZj0iaHR0cHM6Ly93d3cucWlkYW8xMjMuY29tL2Jicy8iIC8+Cgk8IS0tey9pZn0tLT4KCXJgbmB0PG1ldGEgbmFtZT0iYXBwbGljYXRpb24tbmFtZSIgY29udGVudD0iJF9HWydzZXR0aW5nJ11bJ2JibmFtZSddIiAvPgoJPG1ldGEgbmFtZT0ibXNhcHBsaWNhdGlvbi10b29sdGlwIiBjb250ZW50PSIkX0dbJ3NldHRpbmcnXVsnYmJuYW1lJ10iIC8+Cgk8IS0te2lmICRfR1snc2V0dGluZyddWydwb3J0YWxzdGF0dXMnXX0tLT48bWV0YSBuYW1lPSJtc2FwcGxpY2F0aW9uLXRhc2siIGNvbnRlbnQ9Im5hbWU9JF9HWydzZXR0aW5nJ11bJ25hdnMnXVsxXVsnbmF2bmFtZSddO2FjdGlvbi11cmk9e2VjaG8gIWVtcHR5KCRfR1snc2V0dGluZyddWydkb21haW4nXVsnYXBwJ11bJ3BvcnRhbCddKSA/ICRfR1snc2NoZW1lJ10uJzovLycuJF9HWydzZXR0aW5nJ11bJ2RvbWFpbiddWydhcHAnXVsncG9ydGFsJ10gOiAkX0dbc2l0ZXVybF0uJ3BvcnRhbC5waHAnfTtpY29uLXVyaT17JF9HW3NpdGV1cmxdfXtJTUdESVJ9L3BvcnRhbC5pY28iIC8+PCEtLXsvaWZ9LS0+Cgk8bWV0YSBuYW1lPSJtc2FwcGxpY2F0aW9uLXRhc2siIGNvbnRlbnQ9Im5hbWU9JF9HWydzZXR0aW5nJ11bJ25hdnMnXVsyXVsnbmF2bmFtZSddO2FjdGlvbi11cmk9e2VjaG8gIWVtcHR5KCRfR1snc2V0dGluZyddWydkb21haW4nXVsnYXBwJ11bJ2ZvcnVtJ10pID8gJF9HWydzY2hlbWUnXS4nOi8vJy4kX0dbJ3NldHRpbmcnXVsnZG9tYWluJ11bJ2FwcCddWydmb3J1bSddIDogJF9HW3NpdGV1cmxdLidmb3J1bS5waHAnfTtpY29uLXVyaT17JF9HW3NpdGV1cmxdfXtJTUdESVJ9L2Jicy5pY28iIC8+Cgk8IS0te2lmICRfR1snc2V0dGluZyddWydncm91cHN0YXR1cyddfS0tPjxtZXRhIG5hbWU9Im1zYXBwbGljYXRpb24tdGFzayIgY29udGVudD0ibmFtZT0kX0dbJ3NldHRpbmcnXVsnbmF2cyddWzNdWyduYXZuYW1lJ107YWN0aW9uLXVyaT17ZWNobyAhZW1wdHkoJF9HWydzZXR0aW5nJ11bJ2RvbWFpbiddWydhcHAnXVsnZ3JvdXAnXSkgPyAkX0dbJ3NjaGVtZSddLic6Ly8nLiRfR1snc2V0dGluZyddWydkb21haW4nXVsnYXBwJ11bJ2dyb3VwJ10gOiAkX0dbc2l0ZXVybF0uJ2dyb3VwLnBocCd9O2ljb24tdXJpPXskX0dbc2l0ZXVybF19e0lNR0RJUn0vZ3JvdXAuaWNvIiAvPjwhLS17L2lmfS0tPgoJPCEtLXtpZiBoZWxwZXJfYWNjZXNzOjpjaGVja19tb2R1bGUoJ2ZlZWQnKX0tLT48bWV0YSBuYW1lPSJtc2FwcGxpY2F0aW9uLXRhc2siIGNvbnRlbnQ9Im5hbWU9JF9HWydzZXR0aW5nJ11bJ25hdnMnXVs0XVsnbmF2bmFtZSddO2FjdGlvbi11cmk9e2VjaG8gIWVtcHR5KCRfR1snc2V0dGluZyddWydkb21haW4nXVsnYXBwJ11bJ2hvbWUnXSkgPyAkX0dbJ3NjaGVtZSddLic6Ly8nLiRfR1snc2V0dGluZyddWydkb21haW4nXVsnYXBwJ11bJ2hvbWUnXSA6ICRfR1tzaXRldXJsXS4naG9tZS5waHAnfTtpY29uLXVyaT17JF9HW3NpdGV1cmxdfXtJTUdESVJ9L2hvbWUuaWNvIiAvPjwhLS17L2lmfS0tPgoJPCEtLXtpZiAkX0dbJ2Jhc2VzY3JpcHQnXSA9PSAnZm9ydW0nICYmICRfR1snc2V0dGluZyddWydhcmNoaXZlciddfS0tPgoJCTxsaW5rIHJlbD0iYXJjaGl2ZXMiIHRpdGxlPSIkX0dbJ3NldHRpbmcnXVsnYmJuYW1lJ10iIGhyZWY9InskX0dbc2l0ZXVybF19YXJjaGl2ZXIvIiAvPgoJPCEtLXsvaWZ9LS0+Cgk8IS0te2lmICFlbXB0eSgkcnNzaGVhZCl9LS0+JHJzc2hlYWQ8IS0tey9pZn0tLT4KCTwhLS17aWYgd2lkdGhhdXRvKCl9LS0+CgkJPGxpbmsgcmVsPSJzdHlsZXNoZWV0IiBpZD0iY3NzX3dpZHRoYXV0byIgdHlwZT0idGV4dC9jc3MiIGhyZWY9J3skX0dbJ3NldHRpbmcnXVsnY3NzcGF0aCddfXtTVFlMRUlEfV93aWR0aGF1dG8uY3NzP3tWRVJIQVNIfScgLz4KCQk8c2NyaXB0IHR5cGU9InRleHQvamF2YXNjcmlwdCI+SFRNTE5PREUuY2xhc3NOYW1lICs9ICcgd2lkdGhhdXRvJzwvc2NyaXB0PgoJPCEtLXsvaWZ9LS0+Cgk8IS0te2lmICRfR1snYmFzZXNjcmlwdCddID09ICdmb3J1bScgfHwgJF9HWydiYXNlc2NyaXB0J10gPT0gJ2dyb3VwJ30tLT4KCQk8c2NyaXB0IHR5cGU9InRleHQvamF2YXNjcmlwdCIgc3JjPSJ7JF9HW3NldHRpbmddW2pzcGF0aF19Zm9ydW0uanM/e1ZFUkhBU0h9Ij48L3NjcmlwdD4KCTwhLS17ZWxzZWlmICRfR1snYmFzZXNjcmlwdCddID09ICdob21lJ30tLT4KCQk8c2NyaXB0IHR5cGU9InRleHQvamF2YXNjcmlwdCIgc3JjPSJ7JF9HW3NldHRpbmddW2pzcGF0aF19aG9tZS5qcz97VkVSSEFTSH0iPjwvc2NyaXB0PgoJPCEtLXtlbHNlaWYgJF9HWydiYXNlc2NyaXB0J10gPT0gJ3BvcnRhbCd9LS0+CgkJPHNjcmlwdCB0eXBlPSJ0ZXh0L2phdmFzY3JpcHQiIHNyYz0ieyRfR1tzZXR0aW5nXVtqc3BhdGhdfXBvcnRhbC5qcz97VkVSSEFTSH0iPjwvc2NyaXB0PgoJPCEtLXsvaWZ9LS0+Cgk8IS0te2lmICRfR1snYmFzZXNjcmlwdCddICE9ICdwb3J0YWwnICYmICRfR0VUWydkaXknXSA9PSAneWVzJyAmJiBjaGVja19kaXlfcGVybSgkdG9waWMpfS0tPgoJCTxzY3JpcHQgdHlwZT0idGV4dC9qYXZhc2NyaXB0IiBzcmM9InskX0dbc2V0dGluZ11banNwYXRoXX1wb3J0YWwuanM/e1ZFUkhBU0h9Ij48L3NjcmlwdD4KCTwhLS17L2lmfS0tPgoJPCEtLXtpZiAkX0dFVFsnZGl5J10gPT0gJ3llcycgJiYgY2hlY2tfZGl5X3Blcm0oJHRvcGljKX0tLT4KCQk8bGluayByZWw9InN0eWxlc2hlZXQiIHR5cGU9InRleHQvY3NzIiBpZD0iZGl5X2NvbW1vbiIgaHJlZj0ieyRfR1snc2V0dGluZyddWydjc3NwYXRoJ119e1NUWUxFSUR9X2Nzc19kaXkuY3NzP3tWRVJIQVNIfSIgLz4KCTwhLS17L2lmfS0tPgo8L2hlYWQ+Cgo8Ym9keSBpZD0ibnZfeyRfR1tiYXNlc2NyaXB0XX0iIGNsYXNzPSJwZ197Q1VSTU9EVUxFfXtpZiAkX0dbJ2Jhc2VzY3JpcHQnXSA9PT0gJ3BvcnRhbCcgJiYgQ1VSTU9EVUxFID09PSAnbGlzdCcgJiYgIWVtcHR5KCRjYXQpfSB7JGNhdFsnYm9keWNzcyddfXsvaWZ9IiBvbmtleWRvd249ImlmKGV2ZW50LmtleUNvZGU9PTI3KSByZXR1cm4gZmFsc2U7Ij4KCTxkaXYgaWQ9ImFwcGVuZF9wYXJlbnQiPjwvZGl2PjxkaXYgaWQ9ImFqYXh3YWl0aWQiPjwvZGl2PgoJPCEtLXtpZiAkX0dFVFsnZGl5J10gPT0gJ3llcycgJiYgY2hlY2tfZGl5X3Blcm0oJHRvcGljKX0tLT4KCQk8IS0te3RlbXBsYXRlIGNvbW1vbi9oZWFkZXJfZGl5fS0tPgoJPCEtLXsvaWZ9LS0+Cgk8IS0te2lmIGNoZWNrX2RpeV9wZXJtKCR0b3BpYyl9LS0+CgkJPCEtLXt0ZW1wbGF0ZSBjb21tb24vaGVhZGVyX2RpeW5hdn0tLT4KCTwhLS17L2lmfS0tPgoJPCEtLXtpZiBDVVJNT0RVTEUgPT0gJ3RvcGljJyAmJiAkdG9waWMgJiYgZW1wdHkoJHRvcGljWyd1c2VoZWFkZXInXSkgJiYgY2hlY2tfZGl5X3Blcm0oJHRvcGljKX0tLT4KCQkkZGl5bmF2Cgk8IS0tey9pZn0tLT4KCTwhLS17aWYgZW1wdHkoJHRvcGljKSB8fCAkdG9waWNbJ3VzZWhlYWRlciddfS0tPgoJCTwhLS17aWYgJF9HWydzZXR0aW5nJ11bJ21vYmlsZSddWydhbGxvd21vYmlsZSddICYmICghJF9HWydzZXR0aW5nJ11bJ2NhY2hlaW5kZXhsaWZlJ10gJiYgISRfR1snc2V0dGluZyddWydjYWNoZXRocmVhZG9uJ10gfHwgJF9HWyd1aWQnXSkgJiYgKCRfR0VUWydkaXknXSAhPSAneWVzJyB8fCAhJF9HRVRbJ2luYWpheCddKSAmJiAoJF9HWydtb2JpbGUnXSAhPSAnJyAmJiAkX0dbJ2Nvb2tpZSddWydtb2JpbGUnXSA9PSAnJyAmJiAkX0dFVFsnbW9iaWxlJ10gIT0gJ25vJyl9LS0+CgkJCTxkaXYgY2xhc3M9InhpMSBibSBibV9jIj4KCQkJICAgIHtsYW5nIHlvdXJfbW9iaWxlX2Jyb3dzZXJ9PGEgaHJlZj0ieyRfR1snc2l0ZXVybCddfWZvcnVtLnBocD9tb2JpbGU9eWVzIj57bGFuZyBnb190b19tb2JpbGV9PC9hPiA8c3BhbiBjbGFzcz0ieGcxIj58PC9zcGFuPiA8YSBocmVmPSIkX0dbJ3NldHRpbmcnXVsnbW9iaWxlJ11bJ25vbW9iaWxldXJsJ10iPntsYW5nIHRvX2JlX2NvbnRpbnVlfTwvYT4KCQkJPC9kaXY+CgkJPCEtLXsvaWZ9LS0+CgkJPCEtLXtpZiAhZW1wdHkoJF9HWydzZXR0aW5nJ11bJ3Nob3J0Y3V0J10pICYmICRfR1snbWVtYmVyJ11bY3JlZGl0c10gPj0gJF9HWydzZXR0aW5nJ11bJ3Nob3J0Y3V0J119LS0+CgkJCTxkaXYgaWQ9InNob3J0Y3V0Ij4KCQkJCTxzcGFuPjxhIGhyZWY9ImphdmFzY3JpcHQ6OyIgaWQ9InNob3J0Y3V0Y2xvc2VpZCIgdGl0bGU9IntsYW5nIGNsb3NlfSI+e2xhbmcgY2xvc2V9PC9hPjwvc3Bhbj4KCQkJCXtsYW5nIHNob3J0Y3V0X25vdGljZX0KCQkJCTxhIGhyZWY9ImphdmFzY3JpcHQ6OyIgaWQ9InNob3J0Y3V0dGlwIj57bGFuZyBzaG9ydGN1dF9hZGR9PC9hPgoKCQkJPC9kaXY+CgkJCTxzY3JpcHQgdHlwZT0idGV4dC9qYXZhc2NyaXB0Ij5zZXRUaW1lb3V0KHNldFNob3J0Y3V0LCAyMDAwKTs8L3NjcmlwdD4KCQk8IS0tey9pZn0tLT4KCQk8ZGl2IGlkPSJ0b3B0YiIgY2xhc3M9ImNsIj4KCQkJPCEtLXtob29rL2dsb2JhbF9jcG5hdl90b3B9LS0+CgkJCTxkaXYgY2xhc3M9IndwIj4KCQkJCTxkaXYgY2xhc3M9InoiPgoJCQkJCTwhLS17bG9vcCAkX0dbJ3NldHRpbmcnXVsndG9wbmF2cyddWzBdICRuYXZ9LS0+CgkJCQkJCTwhLS17aWYgaXNfYXJyYXkoJG5hdikgJiYgJG5hdlsnYXZhaWxhYmxlJ10gJiYgKCEkbmF2WydsZXZlbCddIHx8ICgkbmF2WydsZXZlbCddID09IDEgJiYgJF9HWyd1aWQnXSkgfHwgKCRuYXZbJ2xldmVsJ10gPT0gMiAmJiAkX0dbJ2FkbWluaWQnXSA+IDApIHx8ICgkbmF2WydsZXZlbCddID09IDMgJiYgJF9HWydhZG1pbmlkJ10gPT0gMSkpfS0tPiRuYXZbY29kZV08IS0tey9pZn0tLT4KCQkJCQk8IS0tey9sb29wfS0tPgoJCQkJCTwhLS17aG9vay9nbG9iYWxfY3BuYXZfZXh0cmExfS0tPgoJCQkJPC9kaXY+CgkJCQk8ZGl2IGNsYXNzPSJ5Ij4KCQkJCQk8YSBpZD0ic3dpdGNoYmxpbmQiIGhyZWY9ImphdmFzY3JpcHQ6OyIgb25jbGljaz0idG9nZ2xlQmxpbmQodGhpcykiIHRpdGxlPSJ7bGFuZyBzd2l0Y2hfYmxpbmR9IiBjbGFzcz0ic3dpdGNoYmxpbmQiPjwvYT4KCQkJCQk8IS0te2hvb2svZ2xvYmFsX2NwbmF2X2V4dHJhMn0tLT4KCQkJCQk8IS0te2xvb3AgJF9HWydzZXR0aW5nJ11bJ3RvcG5hdnMnXVsxXSAkbmF2fS0tPgoJCQkJCQk8IS0te2lmIGlzX2FycmF5KCRuYXYpICYmICRuYXZbJ2F2YWlsYWJsZSddICYmICghJG5hdlsnbGV2ZWwnXSB8fCAoJG5hdlsnbGV2ZWwnXSA9PSAxICYmICRfR1sndWlkJ10pIHx8ICgkbmF2WydsZXZlbCddID09IDIgJiYgJF9HWydhZG1pbmlkJ10gPiAwKSB8fCAoJG5hdlsnbGV2ZWwnXSA9PSAzICYmICRfR1snYWRtaW5pZCddID09IDEpKX0tLT4kbmF2W2NvZGVdPCEtLXsvaWZ9LS0+CgkJCQkJPCEtLXsvbG9vcH0tLT4KCQkJCQk8IS0te2lmIGVtcHR5KCRfR1snZGlzYWJsZWR3aWR0aGF1dG8nXSkgJiYgJF9HWydzZXR0aW5nJ11bJ3N3aXRjaHdpZHRoYXV0byddfS0tPgoJCQkJCQk8YSBocmVmPSJqYXZhc2NyaXB0OjsiIGlkPSJzd2l0Y2h3aWR0aCIgb25jbGljaz0id2lkdGhhdXRvKHRoaXMpIiB0aXRsZT0ie2lmIHdpZHRoYXV0bygpfXtsYW5nIHN3aXRjaF9uYXJyb3d9e2Vsc2V9e2xhbmcgc3dpdGNoX3dpZGV9ey9pZn0iIGNsYXNzPSJzd2l0Y2h3aWR0aCI+PCEtLXtpZiB3aWR0aGF1dG8oKX0tLT57bGFuZyBzd2l0Y2hfbmFycm93fTwhLS17ZWxzZX0tLT57bGFuZyBzd2l0Y2hfd2lkZX08IS0tey9pZn0tLT48L2E+CgkJCQkJPCEtLXsvaWZ9LS0+CgkJCQkJPCEtLXtpZiAkX0dbJ3VpZCddICYmICFlbXB0eSgkX0dbJ3N0eWxlJ11bJ2V4dHN0eWxlJ10pfS0tPjxhIGlkPSJzc2xjdCIgaHJlZj0iamF2YXNjcmlwdDo7IiBvbm1vdXNlb3Zlcj0iZGVsYXlTaG93KHRoaXMsIGZ1bmN0aW9uKCkge3Nob3dNZW51KHsnY3RybGlkJzonc3NsY3QnLCdwb3MnOiczNCEnfSl9KTsiPntsYW5nIGNoYW5nZXN0eWxlfTwvYT48IS0tey9pZn0tLT4KCQkJCQk8IS0te2lmIGNoZWNrX2RpeV9wZXJtKCR0b3BpYyl9LS0+CgkJCQkJCSRkaXluYXYKCQkJCQk8IS0tey9pZn0tLT4KCQkJCTwvZGl2PgoJCQk8L2Rpdj4KCQk8L2Rpdj4KCgkJPCEtLXtpZiAhSVNfUk9CT1R9LS0+CgkJCTwhLS17aWYgJF9HWyd1aWQnXSAmJiAhJF9HWydzZXR0aW5nJ11bJ2JiY2xvc2VkJ10gJiYgZW1wdHkoJF9HWydtZW1iZXInXVsnZnJlZXplJ10pICYmICRfR1snbWVtYmVyJ11bJ2dyb3VwaWQnXSAhPSA1fS0tPgoJCQk8dWwgaWQ9Im15cHJvbXB0X21lbnUiIGNsYXNzPSJwX3BvcCIgc3R5bGU9ImRpc3BsYXk6IG5vbmU7Ij4KCQkJCTxsaT48YSBocmVmPSJob21lLnBocD9tb2Q9c3BhY2UmZG89cG0iIGlkPSJwbV9udGMiIHN0eWxlPSJiYWNrZ3JvdW5kLXJlcGVhdDogbm8tcmVwZWF0OyBiYWNrZ3JvdW5kLXBvc2l0aW9uOiAwIDUwJTsiPjxlbSBjbGFzcz0icHJvbXB0X25ld3N7aWYgZW1wdHkoJF9HW21lbWJlcl1bbmV3cG1dKX1fMHsvaWZ9Ij48L2VtPntsYW5nIHBtX2NlbnRlcn08L2E+PC9saT4KCQkJCTwhLS17aWYgJF9HWydzZXR0aW5nJ11bJ2ZvbGxvd3N0YXR1cyddfS0tPgoJCQkJCTxsaT48YSBocmVmPSJob21lLnBocD9tb2Q9Zm9sbG93JmRvPWZvbGxvd2VyIj48ZW0gY2xhc3M9InByb21wdF9mb2xsb3dlcntpZiBlbXB0eSgkX0dbbWVtYmVyXVtuZXdwcm9tcHRfbnVtXVtmb2xsb3dlcl0pfV8wey9pZn0iPjwvZW0+PCEtLXtsYW5nIG5vdGljZV9pbnRlcmFjdGl2ZV9mb2xsb3dlcn0tLT57aWYgJF9HW21lbWJlcl1bbmV3cHJvbXB0X251bV1bZm9sbG93ZXJdfSgkX0dbbWVtYmVyXVtuZXdwcm9tcHRfbnVtXVtmb2xsb3dlcl0pey9pZn08L2E+PC9saT4KCQkJCQk8IS0te2lmICRfR1ttZW1iZXJdW25ld3Byb21wdF0gJiYgJF9HW21lbWJlcl1bbmV3cHJvbXB0X251bV1bZm9sbG93XX0tLT4KCQkJCQkJPGxpPjxhIGhyZWY9ImhvbWUucGhwP21vZD1mb2xsb3ciPjxlbSBjbGFzcz0icHJvbXB0X2NvbmNlcm4iPjwvZW0+PCEtLXtsYW5nIG5vdGljZV9pbnRlcmFjdGl2ZV9mb2xsb3d9LS0+KCRfR1ttZW1iZXJdW25ld3Byb21wdF9udW1dW2ZvbGxvd10pPC9hPjwvbGk+CgkJCQkJPCEtLXsvaWZ9LS0+CgkJCQk8IS0tey9pZn0tLT4KCQkJCTwhLS17aWYgJF9HW21lbWJlcl1bbmV3cHJvbXB0XX0tLT4KCQkJCQk8IS0te2xvb3AgJF9HWydtZW1iZXInXVsnY2F0ZWdvcnlfbnVtJ10gJGtleSAkdmFsfS0tPgoJCQkJCQk8bGk+PGEgaHJlZj0iaG9tZS5waHA/bW9kPXNwYWNlJmRvPW5vdGljZSZ2aWV3PSRrZXkiPjxlbSBjbGFzcz0ibm90aWNlXyRrZXkiPjwvZW0+PCEtLXtlY2hvIGxhbmcoJ3RlbXBsYXRlJywgJ25vdGljZV8nLiRrZXkpfS0tPig8c3BhbiBjbGFzcz0icnEiPiR2YWw8L3NwYW4+KTwvYT48L2xpPgoJCQkJCTwhLS17L2xvb3B9LS0+CgkJCQk8IS0tey9pZn0tLT4KCQkJCTwhLS17aWYgZW1wdHkoJF9HWydjb29raWUnXVsnaWdub3JlX25vdGljZSddKX0tLT4KCQkJCQk8bGkgY2xhc3M9Imlnbm9yZV9ub3RpY2VsaSI+PGEgaHJlZj0iamF2YXNjcmlwdDo7IiBvbmNsaWNrPSJzZXRjb29raWUoJ2lnbm9yZV9ub3RpY2UnLCAxKTtoaWRlTWVudSgnbXlwcm9tcHRfbWVudScpIiB0aXRsZT0ie2xhbmcgdGVtcG9yYXJpbHlfdG9fcmVtaW5kfSI+PGVtIGNsYXNzPSJpZ25vcmVfbm90aWNlIj48L2VtPjwvYT48L2xpPgoJCQkJPCEtLXsvaWZ9LS0+CgkJCTwvdWw+CgkJCTwhLS17L2lmfS0tPgoJCQk8IS0te2lmICRfR1sndWlkJ10gJiYgIWVtcHR5KCRfR1snc3R5bGUnXVsnZXh0c3R5bGUnXSl9LS0+CgkJCQk8ZGl2IGlkPSJzc2xjdF9tZW51IiBjbGFzcz0iY2wgcF9wb3AiIHN0eWxlPSJkaXNwbGF5OiBub25lOyI+CgkJCQkJPCEtLXtpZiAhJF9HW3N0eWxlXVtkZWZhdWx0ZXh0c3R5bGVdfS0tPjxzcGFuIGNsYXNzPSJzc2xjdF9idG4iIG9uY2xpY2s9ImV4dHN0eWxlKCcnKSIgdGl0bGU9IntsYW5nIGRlZmF1bHR9Ij48aT48L2k+PC9zcGFuPjwhLS17L2lmfS0tPgoJCQkJCTwhLS17bG9vcCAkX0dbJ3N0eWxlJ11bJ2V4dHN0eWxlJ10gJGV4dHN0eWxlfS0tPgoJCQkJCQk8c3BhbiBjbGFzcz0ic3NsY3RfYnRuIiBvbmNsaWNrPSJleHRzdHlsZSgnJGV4dHN0eWxlWzBdJykiIHRpdGxlPSIkZXh0c3R5bGVbMV0iPjxpIHN0eWxlPSdiYWNrZ3JvdW5kOiRleHRzdHlsZVsyXSc+PC9pPjwvc3Bhbj4KCQkJCQk8IS0tey9sb29wfS0tPgoJCQkJPC9kaXY+CgkJCTwhLS17L2lmfS0tPgoJCQk8IS0te2lmICRfR1sndWlkJ119LS0+CgkJCQk8dWwgaWQ9Im15aXRlbV9tZW51IiBjbGFzcz0icF9wb3AiIHN0eWxlPSJkaXNwbGF5OiBub25lOyI+CgkJCQkJPCEtLXtpZiAkX0dbJ3NldHRpbmcnXVsnZm9ydW1zdGF0dXMnXX0tLT48bGk+PGEgaHJlZj0iaG9tZS5waHA/bW9kPXNwYWNlJmRvPXRocmVhZCZ2aWV3PW1lIj57bGFuZyBteXBvc3R9PC9hPjwvbGk+PCEtLXsvaWZ9LS0+CgkJCQkJPCEtLXtpZiAkX0dbJ3NldHRpbmcnXVsnZmF2b3JpdGVzdGF0dXMnXX0tLT48bGk+PGEgaHJlZj0iaG9tZS5waHA/bW9kPXNwYWNlJmRvPWZhdm9yaXRlJnZpZXc9bWUiPntsYW5nIGZhdm9yaXRlfTwvYT48L2xpPjwhLS17L2lmfS0tPgoJCQkJCTwhLS17aWYgJF9HWydzZXR0aW5nJ11bJ2ZyaWVuZHN0YXR1cyddfS0tPjxsaT48YSBocmVmPSJob21lLnBocD9tb2Q9c3BhY2UmZG89ZnJpZW5kIj57bGFuZyBmcmllbmRzfTwvYT48L2xpPjwhLS17L2lmfS0tPgoJCQkJCTwhLS17aG9vay9nbG9iYWxfbXlpdGVtX2V4dHJhfS0tPgoJCQkJPC91bD4KCQkJPCEtLXsvaWZ9LS0+CgkJCTwhLS17c3VidGVtcGxhdGUgY29tbW9uL2hlYWRlcl9xbWVudX0tLT4KCQk8IS0tey9pZn0tLT4KCgkJPCEtLXthZC9oZWFkZXJiYW5uZXIvd3AgYV9ofS0tPgoJCTxkaXYgaWQ9ImhkIj4KCQkJPGRpdiBjbGFzcz0id3AiPgoJCQkJPGRpdiBjbGFzcz0iaGRjIGNsIj4KCQkJCQk8IS0te2V2YWwgJG1uaWQgPSBnZXRjdXJyZW50bmF2KCk7fS0tPgoJCQkJCTxoMj48IS0te2lmICFpc3NldCgkX0dbJ3NldHRpbmcnXVsnbmF2bG9nb3MnXVskbW5pZF0pfS0tPjxhIGhyZWY9IntpZiAkX0dbJ3NldHRpbmcnXVsnZG9tYWluJ11bJ2FwcCddWydkZWZhdWx0J119eyRfR1snc2NoZW1lJ119Oi8veyRfR1snc2V0dGluZyddWydkb21haW4nXVsnYXBwJ11bJ2RlZmF1bHQnXX0ve2Vsc2V9Li97L2lmfSIgdGl0bGU9IiRfR1snc2V0dGluZyddWydiYm5hbWUnXSI+eyRfR1snc3R5bGUnXVsnYm9hcmRsb2dvJ119PC9hPjwhLS17ZWxzZX0tLT4kX0dbJ3NldHRpbmcnXVsnbmF2bG9nb3MnXVskbW5pZF08IS0tey9pZn0tLT48L2gyPgoJCQkJCTwhLS17dGVtcGxhdGUgY29tbW9uL2hlYWRlcl91c2Vyc3RhdHVzfS0tPgoJCQkJPC9kaXY+CgoJCQkJPGRpdiBpZD0ibnYiPgoJCQkJCTxhIGhyZWY9ImphdmFzY3JpcHQ6OyIgaWQ9InFtZW51IiBvbm1vdXNlb3Zlcj0iZGVsYXlTaG93KHRoaXMsIGZ1bmN0aW9uICgpIHtzaG93TWVudSh7J2N0cmxpZCc6J3FtZW51JywncG9zJzonMzQhJywnY3RybGNsYXNzJzonYScsJ2R1cmF0aW9uJzoyfSk7c2hvd0ZvcnVtbWVudSgkX0dbZmlkXSk7fSkiPntsYW5nIG15X25hdn08L2E+CgkJCQkJPHVsPgoJCQkJCQk8IS0te2xvb3AgJF9HWydzZXR0aW5nJ11bJ25hdnMnXSAkbmF2fS0tPgoJCQkJCQkJPCEtLXtpZiBpc19hcnJheSgkbmF2KSAmJiAkbmF2WydhdmFpbGFibGUnXSAmJiAoISRuYXZbJ2xldmVsJ10gfHwgKCRuYXZbJ2xldmVsJ10gPT0gMSAmJiAkX0dbJ3VpZCddKSB8fCAoJG5hdlsnbGV2ZWwnXSA9PSAyICYmICRfR1snYWRtaW5pZCddID4gMCkgfHwgKCRuYXZbJ2xldmVsJ10gPT0gMyAmJiAkX0dbJ2FkbWluaWQnXSA9PSAxKSl9LS0+PGxpIHtpZiAkbW5pZCA9PSAkbmF2W25hdmlkXSB8fCBzdWJzdHIoJF9TRVJWRVJbJ1JFUVVFU1RfVVJJJ10sIDEpID09IHN0cl9yZXBsYWNlKCcuLycsICcnLCAkbmF2W2ZpbGVuYW1lXSl9Y2xhc3M9ImEiIHsvaWZ9JG5hdltuYXZdPjwvbGk+PCEtLXsvaWZ9LS0+CgkJCQkJCTwhLS17L2xvb3B9LS0+CgkJCQkJPC91bD4KCQkJCQk8IS0te2hvb2svZ2xvYmFsX25hdl9leHRyYX0tLT4KCQkJCTwvZGl2PgoJCQkJPCEtLXtpZiAhZW1wdHkoJF9HWydzZXR0aW5nJ11bJ3BsdWdpbnMnXVsnanNtZW51J10pfS0tPgoJCQkJCTx1bCBjbGFzcz0icF9wb3AgaF9wb3AiIGlkPSJwbHVnaW5fbWVudSIgc3R5bGU9ImRpc3BsYXk6IG5vbmUiPgoJCQkJCTwhLS17bG9vcCAkX0dbJ3NldHRpbmcnXVsncGx1Z2lucyddWydqc21lbnUnXSAkbW9kdWxlfS0tPgoJCQkJCQkgPCEtLXtpZiBpbl9hcnJheSgkbW9kdWxlWydhZG1pbmlkJ10sIGFycmF5KDAsIC0xKSkgfHwgKCRtb2R1bGVbJ2FkbWluaWQnXSAmJiAkX0dbJ2FkbWluaWQnXSA+IDAgJiYgJG1vZHVsZVsnYWRtaW5pZCddID49ICRfR1snYWRtaW5pZCddKX0tLT4KCQkJCQkJIDxsaT4kbW9kdWxlW3VybF08L2xpPgoJCQkJCQkgPCEtLXsvaWZ9LS0+CgkJCQkJPCEtLXsvbG9vcH0tLT4KCQkJCQk8L3VsPgoJCQkJPCEtLXsvaWZ9LS0+CgkJCQkkX0dbc2V0dGluZ11bbWVudW5hdnNdCgkJCQk8ZGl2IGlkPSJtdSIgY2xhc3M9ImNsIj4KCQkJCTwhLS17aWYgJF9HWydzZXR0aW5nJ11bJ3N1Ym5hdnMnXX0tLT4KCQkJCQk8IS0te2xvb3AgJF9HW3NldHRpbmddW3N1Ym5hdnNdICRuYXZpZCAkc3VibmF2fS0tPgoJCQkJCQk8IS0te2lmICRfR1snc2V0dGluZyddWyduYXZzdWJob3ZlciddIHx8ICRtbmlkID09ICRuYXZpZH0tLT4KCQkJCQkJPHVsIGNsYXNzPSJjbCB7aWYgJG1uaWQgPT0gJG5hdmlkfWN1cnJlbnR7L2lmfSIgaWQ9InNuYXZfJG5hdmlkIntpZiAkbW5pZCAhPSAkbmF2aWR9IHN0eWxlPSJkaXNwbGF5Om5vbmUiey9pZn0+CgkJCQkJCSRzdWJuYXYKCQkJCQkJPC91bD4KCQkJCQkJPCEtLXsvaWZ9LS0+CgkJCQkJPCEtLXsvbG9vcH0tLT4KCQkJCTwhLS17L2lmfS0tPgoJCQkJPC9kaXY+CgkJCQk8IS0te2FkL3N1Ym5hdmJhbm5lci9hX211fS0tPgoJCQkJPCEtLXtzdWJ0ZW1wbGF0ZSBjb21tb24vcHVic2VhcmNoZm9ybX0tLT4KCQkJPC9kaXY+CgkJPC9kaXY+CgoJCTwhLS17aG9vay9nbG9iYWxfaGVhZGVyfS0tPgoJPCEtLXsvaWZ9LS0+CgoJPGRpdiBpZD0id3AiIGNsYXNzPSJ3cCI+Cg==
qidao123.com ToB IT社区-企服评测·应用市场»论坛 › 容器及微服务 › 容器及微服务 › k8s单机容器网络(20250216)
返回列表 发新帖

k8s单机容器网络(20250216)

[复制链接]
发表于 2025-2-17 18:51:43 | 显示全部楼层 |阅读模式

马上注册，结交更多好友，享用更多功能，让你轻松玩转社区。

您需要 登录 才可以下载或查看，没有账号？立即注册

×
k8s单机容器网络(20250216)

​        Linux 容器能瞥见的“网络栈”，现实上是被隔离在它自己的 Network Namespace 当中的。
​        “网络栈”，就包罗了：网卡（Network Interface）、回环设备（Loopback Device）、路由表（Routing Table）和 iptables 规则。
Veth Pair 设备

​        Veth Pair 设备的特点是：它被创建出来后，总是以两张虚拟网卡（Veth Peer）的形式成对出现的。而且，从其中一个“网卡”发出的数据包，可以直接出现在与它对应的另一张“网卡”上，哪怕这两个“网卡”在差别的 Network Namespace 里。
  1. Microsoft Windows [版本 10.0.26100.2894]
  2. (c) Microsoft Corporation。保留所有权利。
  4. C:\Users\admin>ssh root@192.168.117.207
  5. root@192.168.117.207's password:
  6. Last login: Mon Feb 17 08:19:32 2025
  7. [root@k8s-master ~]# docker ps
  8. CONTAINER ID   IMAGE                                               COMMAND                  CREATED          STATUS          PORTS     NAMES
  9. 1ee1263b4193   cbb01a7bd410                                        "/coredns -conf /etc…"   1 second ago     Up 1 second               k8s_coredns_coredns-857d9ff4c9-29ldj_kube-system_9ee2e5e5-d728-4c02-a87e-8dcaab82fbd7_13
  10. 829516e501fa   registry.aliyuncs.com/google_containers/pause:3.8   "/pause"                 3 seconds ago    Up 2 seconds              k8s_POD_coredns-857d9ff4c9-29ldj_kube-system_9ee2e5e5-d728-4c02-a87e-8dcaab82fbd7_8
  11. e0c8a6330d0e   9344fce2372f                                        "/usr/local/bin/kube…"   7 seconds ago    Up 6 seconds              k8s_kube-proxy_kube-proxy-nq4x2_kube-system_a3ee7cb5-f97d-4339-8f9e-01e0e15874ba_9
  12. 255fea7d86a5   registry.aliyuncs.com/google_containers/pause:3.8   "/pause"                 8 seconds ago    Up 7 seconds              k8s_POD_calico-node-9fhpq_kube-system_92a3a119-8007-48a9-8743-0afdf65f592c_7
  13. 36c5922e79eb   registry.aliyuncs.com/google_containers/pause:3.8   "/pause"                 9 seconds ago    Up 8 seconds              k8s_POD_kube-proxy-nq4x2_kube-system_a3ee7cb5-f97d-4339-8f9e-01e0e15874ba_7
  14. 1cfe981dc26a   a0eed15eed44                                        "etcd --advertise-cl…"   23 seconds ago   Up 23 seconds             k8s_etcd_etcd-k8s-master_kube-system_e4b42e5b51c6629d934233cc43f26a22_9
  15. 17717a8530ef   6fc5e6b7218c                                        "kube-scheduler --au…"   23 seconds ago   Up 23 seconds             k8s_kube-scheduler_kube-scheduler-k8s-master_kube-system_299cca9182c20d90f643981b13c43213_16
  16. e0df13dfff62   8a9000f98a52                                        "kube-apiserver --ad…"   24 seconds ago   Up 23 seconds             k8s_kube-apiserver_kube-apiserver-k8s-master_kube-system_bc05f019b265f704d6a2ffb204a2c88f_10
  17. 6a21496a57a4   138fb5a3a2e3                                        "kube-controller-man…"   24 seconds ago   Up 23 seconds             k8s_kube-controller-manager_kube-controller-manager-k8s-master_kube-system_51eafc84967051e22b58cf0ebce14e35_15
  18. 5631104357a5   registry.aliyuncs.com/google_containers/pause:3.8   "/pause"                 25 seconds ago   Up 25 seconds             k8s_POD_kube-apiserver-k8s-master_kube-system_bc05f019b265f704d6a2ffb204a2c88f_7
  19. 562543f7a8d6   registry.aliyuncs.com/google_containers/pause:3.8   "/pause"                 25 seconds ago   Up 25 seconds             k8s_POD_kube-controller-manager-k8s-master_kube-system_51eafc84967051e22b58cf0ebce14e35_7
  20. 16dbdd75513f   registry.aliyuncs.com/google_containers/pause:3.8   "/pause"                 25 seconds ago   Up 25 seconds             k8s_POD_kube-scheduler-k8s-master_kube-system_299cca9182c20d90f643981b13c43213_7
  21. 5bfab6a1a042   registry.aliyuncs.com/google_containers/pause:3.8   "/pause"                 26 seconds ago   Up 25 seconds             k8s_POD_etcd-k8s-master_kube-system_e4b42e5b51c6629d934233cc43f26a22_7
  22. [root@k8s-master ~]# docker start nginx-1
  23. nginx-1
  24. [root@k8s-master ~]# docker ps
  25. CONTAINER ID   IMAGE                                               COMMAND                  CREATED              STATUS              PORTS     NAMES
  26. 063af5a1782b   17e960f4e39c                                        "start_runit"            12 seconds ago       Up 12 seconds                 k8s_calico-node_calico-node-9fhpq_kube-system_92a3a119-8007-48a9-8743-0afdf65f592c_66
  27. 133fda8d5c2f   cbb01a7bd410                                        "/coredns -conf /etc…"   22 seconds ago       Up 21 seconds                 k8s_coredns_coredns-857d9ff4c9-ntrmg_kube-system_9a07dc52-b60a-4376-add2-5a128335c9df_12
  28. 2cad37aaa64d   08c1b67c88ce                                        "/usr/bin/kube-contr…"   22 seconds ago       Up 22 seconds                 k8s_calico-kube-controllers_calico-kube-controllers-558d465845-x59c8_kube-system_1586cb4f-6051-4cf2-bcbc-7a05f93739ee_11
  29. 245ed185ea4a   registry.aliyuncs.com/google_containers/pause:3.8   "/pause"                 27 seconds ago       Up 26 seconds                 k8s_POD_coredns-857d9ff4c9-ntrmg_kube-system_9a07dc52-b60a-4376-add2-5a128335c9df_8
  30. 60a93585eea1   registry.aliyuncs.com/google_containers/pause:3.8   "/pause"                 27 seconds ago       Up 27 seconds                 k8s_POD_calico-kube-controllers-558d465845-x59c8_kube-system_1586cb4f-6051-4cf2-bcbc-7a05f93739ee_9
  31. 1ee1263b4193   cbb01a7bd410                                        "/coredns -conf /etc…"   45 seconds ago       Up 45 seconds                 k8s_coredns_coredns-857d9ff4c9-29ldj_kube-system_9ee2e5e5-d728-4c02-a87e-8dcaab82fbd7_13
  32. 829516e501fa   registry.aliyuncs.com/google_containers/pause:3.8   "/pause"                 47 seconds ago       Up 46 seconds                 k8s_POD_coredns-857d9ff4c9-29ldj_kube-system_9ee2e5e5-d728-4c02-a87e-8dcaab82fbd7_8
  33. e0c8a6330d0e   9344fce2372f                                        "/usr/local/bin/kube…"   51 seconds ago       Up 50 seconds                 k8s_kube-proxy_kube-proxy-nq4x2_kube-system_a3ee7cb5-f97d-4339-8f9e-01e0e15874ba_9
  34. 255fea7d86a5   registry.aliyuncs.com/google_containers/pause:3.8   "/pause"                 52 seconds ago       Up 51 seconds                 k8s_POD_calico-node-9fhpq_kube-system_92a3a119-8007-48a9-8743-0afdf65f592c_7
  35. 36c5922e79eb   registry.aliyuncs.com/google_containers/pause:3.8   "/pause"                 53 seconds ago       Up 52 seconds                 k8s_POD_kube-proxy-nq4x2_kube-system_a3ee7cb5-f97d-4339-8f9e-01e0e15874ba_7
  36. 1cfe981dc26a   a0eed15eed44                                        "etcd --advertise-cl…"   About a minute ago   Up About a minute             k8s_etcd_etcd-k8s-master_kube-system_e4b42e5b51c6629d934233cc43f26a22_9
  37. 17717a8530ef   6fc5e6b7218c                                        "kube-scheduler --au…"   About a minute ago   Up About a minute             k8s_kube-scheduler_kube-scheduler-k8s-master_kube-system_299cca9182c20d90f643981b13c43213_16
  38. e0df13dfff62   8a9000f98a52                                        "kube-apiserver --ad…"   About a minute ago   Up About a minute             k8s_kube-apiserver_kube-apiserver-k8s-master_kube-system_bc05f019b265f704d6a2ffb204a2c88f_10
  39. 6a21496a57a4   138fb5a3a2e3                                        "kube-controller-man…"   About a minute ago   Up About a minute             k8s_kube-controller-manager_kube-controller-manager-k8s-master_kube-system_51eafc84967051e22b58cf0ebce14e35_15
  40. 5631104357a5   registry.aliyuncs.com/google_containers/pause:3.8   "/pause"                 About a minute ago   Up About a minute             k8s_POD_kube-apiserver-k8s-master_kube-system_bc05f019b265f704d6a2ffb204a2c88f_7
  41. 562543f7a8d6   registry.aliyuncs.com/google_containers/pause:3.8   "/pause"                 About a minute ago   Up About a minute             k8s_POD_kube-controller-manager-k8s-master_kube-system_51eafc84967051e22b58cf0ebce14e35_7
  42. 16dbdd75513f   registry.aliyuncs.com/google_containers/pause:3.8   "/pause"                 About a minute ago   Up About a minute             k8s_POD_kube-scheduler-k8s-master_kube-system_299cca9182c20d90f643981b13c43213_7
  43. 5bfab6a1a042   registry.aliyuncs.com/google_containers/pause:3.8   "/pause"                 About a minute ago   Up About a minute             k8s_POD_etcd-k8s-master_kube-system_e4b42e5b51c6629d934233cc43f26a22_7
  44. d85077c98a69   nginx                                               "/docker-entrypoint.…"   18 hours ago         Up 12 seconds       80/tcp    nginx-1
复制代码
  1. [root@k8s-master ~]# docker exec -it nginx-1 /bin/bash
  2. root@d85077c98a69:/# ifconfig
  3. eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
  4.         inet 172.17.0.2  netmask 255.255.0.0  broadcast 172.17.255.255
  5.         ether 02:42:ac:11:00:02  txqueuelen 0  (Ethernet)
  6.         RX packets 14  bytes 1252 (1.2 KiB)
  7.         RX errors 0  dropped 0  overruns 0  frame 0
  8.         TX packets 0  bytes 0 (0.0 B)
  9.         TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
  11. lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
  12.         inet 127.0.0.1  netmask 255.0.0.0
  13.         inet6 ::1  prefixlen 128  scopeid 0x10<host>
  14.         loop  txqueuelen 1000  (Local Loopback)
  15.         RX packets 0  bytes 0 (0.0 B)
  16.         RX errors 0  dropped 0  overruns 0  frame 0
  17.         TX packets 0  bytes 0 (0.0 B)
  18.         TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
  20. root@d85077c98a69:/# route
  21. Kernel IP routing table
  22. Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
  23. default         172.17.0.1      0.0.0.0         UG    0      0        0 eth0
  24. 172.17.0.0      0.0.0.0         255.255.0.0     U     0      0        0 eth0
  25. #宿主机
  26. root@d85077c98a69:/# exit
  27. exit
  28. [root@k8s-master ~]# ifconfig
  29. cali6632e2eedff: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
  30.         inet6 fe80::ecee:eeff:feee:eeee  prefixlen 64  scopeid 0x20<link>
  31.         ether ee:ee:ee:ee:ee:ee  txqueuelen 1000  (Ethernet)
  32.         RX packets 3  bytes 125 (125.0 B)
  33.         RX errors 0  dropped 0  overruns 0  frame 0
  34.         TX packets 8  bytes 770 (770.0 B)
  35.         TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
  37. cali7b6489f2f47: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
  38.         inet6 fe80::ecee:eeff:feee:eeee  prefixlen 64  scopeid 0x20<link>
  39.         ether ee:ee:ee:ee:ee:ee  txqueuelen 1000  (Ethernet)
  40.         RX packets 3  bytes 125 (125.0 B)
  41.         RX errors 0  dropped 0  overruns 0  frame 0
  42.         TX packets 8  bytes 770 (770.0 B)
  43.         TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
  45. calieaec58fb34e: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
  46.         inet6 fe80::ecee:eeff:feee:eeee  prefixlen 64  scopeid 0x20<link>
  47.         ether ee:ee:ee:ee:ee:ee  txqueuelen 1000  (Ethernet)
  48.         RX packets 3  bytes 125 (125.0 B)
  49.         RX errors 0  dropped 0  overruns 0  frame 0
  50.         TX packets 8  bytes 770 (770.0 B)
  51.         TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
  53. docker0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
  54.         inet 172.17.0.1  netmask 255.255.0.0  broadcast 172.17.255.255
  55.         inet6 fe80::42:5fff:fe05:698c  prefixlen 64  scopeid 0x20<link>
  56.         ether 02:42:5f:05:69:8c  txqueuelen 0  (Ethernet)
  57.         RX packets 3  bytes 125 (125.0 B)
  58.         RX errors 0  dropped 0  overruns 0  frame 0
  59.         TX packets 8  bytes 770 (770.0 B)
  60.         TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
  62. ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
  63.         inet 192.168.117.207  netmask 255.255.255.0  broadcast 192.168.117.255
  64.         inet6 fe80::20c:29ff:fe96:278c  prefixlen 64  scopeid 0x20<link>
  65.         ether 00:0c:29:96:27:8c  txqueuelen 1000  (Ethernet)
  66.         RX packets 554  bytes 64561 (63.0 KiB)
  67.         RX errors 0  dropped 0  overruns 0  frame 0
  68.         TX packets 596  bytes 65850 (64.3 KiB)
  69.         TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
  71. lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
  72.         inet 127.0.0.1  netmask 255.0.0.0
  73.         inet6 ::1  prefixlen 128  scopeid 0x10<host>
  74.         loop  txqueuelen 1000  (Local Loopback)
  75.         RX packets 49719  bytes 16290594 (15.5 MiB)
  76.         RX errors 0  dropped 0  overruns 0  frame 0
  77.         TX packets 49719  bytes 16290594 (15.5 MiB)
  78.         TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
  80. veth6881202: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
  81.         inet6 fe80::408f:cdff:fe98:623a  prefixlen 64  scopeid 0x20<link>
  82.         ether 42:8f:cd:98:62:3a  txqueuelen 0  (Ethernet)
  83.         RX packets 3  bytes 167 (167.0 B)
  84.         RX errors 0  dropped 0  overruns 0  frame 0
  85.         TX packets 18  bytes 1566 (1.5 KiB)
  86.         TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
  88. [root@k8s-master ~]#
  89. [root@k8s-master ~]# brctl show
  90. bridge name     bridge id               STP enabled     interfaces
  91. docker0         8000.02425f05698c       no              veth6881202
  92. [root@k8s-master ~]#
复制代码
​        这就使得 Veth Pair 常常被用作连接差别 Network Namespace 的“网线”。
​        我们启动了一个叫作 nginx-1 的容器
​        这个容器里有一张叫作 eth0 的网卡，它正是一个 Veth Pair 设备在容器里的这一端。
​        通过 route 命令检察 nginx-1 容器的路由表，我们可以看到，这个 eth0 网卡是这个容器里的默认路由设备；所有对 172.17.0.0/16 网段的请求，也会被交给 eth0 来处理（第二条 172.17.0.0 路由规则）。
​        通过宿主机 ifconfig 命令的输出，你可以看到，nginx-1 容器对应的 Veth Pair 设备，在宿主机上是一张虚拟网卡。它的名字叫作veth6881202
​        而且，通过 brctl show 的输出，你可以看到这张网卡被“插”在了 docker0 上。
假如我们再在这台宿主机上启动另一个 Docker 容器，比如 nginx-2
  1. [root@k8s-master ~]# brctl show
  2. bridge name     bridge id               STP enabled     interfaces
  3. docker0         8000.02425f05698c       no              veth6881202
  4. [root@k8s-master ~]# docker run -d --name nginx-2 nginx
  5. e3b1a33fa82952f99bdf47e1451d05d83a9686cb006798744d2e593f02cf65c8
  6. [root@k8s-master ~]# brctl show
  7. bridge name     bridge id               STP enabled     interfaces
  8. docker0         8000.02425f05698c       no              veth40408f3
  9.                                                         veth6881202
  10. [root@k8s-master ~]#
复制代码
检察容器ip
  1. [root@k8s-master ~]#
  2. [root@k8s-master ~]# docker inspect nginx-1
  3. [
  4.     {
  5.         "Id": "d85077c98a69846efe9bf17c4b1b4efb2152ec2078f5de483edc524c674eed76",
  6.         "Created": "2025-02-16T06:21:15.681636573Z",
  7.         "Path": "/docker-entrypoint.sh",
  8. ----------
  9.    
  10.                    "Links": null,
  11.                     "Aliases": null,
  12.                     "MacAddress": "02:42:ac:11:00:02",
  13.                     "DriverOpts": null,
  14.                     "NetworkID": "5ce1ccec1789844b6a4712acd0c8d6f0ef9fba840c00f53be667a0dd6fbae39c",
  15.                     "EndpointID": "786e7d287ca79fda20dc3895bb64b9830a99f1989538fd503e9f877e4ad574f3",
  16.                     "Gateway": "172.17.0.1",
  17.                     "IPAddress": "172.17.0.2",
  18.                     "IPPrefixLen": 16,
  19.                     "IPv6Gateway": "",
  20.                     "GlobalIPv6Address": "",
  21.                     "GlobalIPv6PrefixLen": 0,
  22.                     "DNSNames": null
  23.                 }
  24.             }
  25.         }
  26.     }
  27. ]
复制代码
ip为"IPAddress": "172.17.0.2",
进入nginx-2来ping nginx-1（curl也行）
  1. [root@k8s-master ~]# docker exec -it nginx-2 /bin/bash
  2. root@e3b1a33fa829:/# ping 172.17.0.2
  3. bash: ping: command not found
  4. root@e3b1a33fa829:/# yum -y install ping
  5. bash: yum: command not found
  6. root@e3b1a33fa829:/# apt-get install -y iputils-ping
  7. Reading package lists... Done
  8. Building dependency tree... Done
  9. Reading state information... Done
  10. E: Unable to locate package iputils-ping
  11. root@e3b1a33fa829:/# curl http://172.17.0.2
  12. <!DOCTYPE html>
  13. <html>
  14. <head>
  15. <title>Welcome to nginx!</title>
  17. </head>
  18. <body>
  19. <h1>Welcome to nginx!</h1>
  20. <p>If you see this page, the nginx web server is successfully installed and
  21. working. Further configuration is required.</p>
  23. <p>For online documentation and support please refer to
  24. <a target="_blank" href="http://nginx.org/">nginx.org</a>.<br/>
  25. Commercial support is available at
  26. <a target="_blank" href="http://nginx.com/">nginx.com</a>.</p>
  28. <p><em>Thank you for using nginx.</em></p>
  29. </body>
  30. </html>
  31. root@e3b1a33fa829:/#
复制代码

​        当你在 nginx-1 容器里访问 nginx-2 容器的 IP 地址（比如 ping 172.17.0.3）的时候，这个目的 IP 地址会匹配到 nginx-1 容器里的第二条路由规则。可以看到，这条路由规则的网关（Gateway）是 0.0.0.0，这就意味着这是一条直连规则，即：凡是匹配到这条规则的 IP 包，应该经过本机的 eth0 网卡，通过二层网络直接发往目的主机。
​        这个 eth0 网卡，是一个 Veth Pair，它的一端在这个 nginx-1 容器的 Network Namespace 里，而另一端则位于宿主机上（Host Namespace），而且被“插”在了宿主机的 docker0 网桥上。
​        一旦一张虚拟网卡被“插”在网桥上，它就会变成该网桥的“从设备”。从设备会被“剥夺”调用网络协议栈处理数据包的资格，从而“降级”成为网桥上的一个端口。而这个端口唯一的作用，就是接收流入的数据包，然后把这些数据包的“生杀大权”（比如转发或者丢弃），全部交给对应的网桥。
​        在收到这些 ARP 请求之后，docker0 网桥就会扮演二层互换机的角色，把 ARP 广播转发到其他被“插”在 docker0 上的虚拟网卡上。这样，同样连接在 docker0 上的 nginx-2 容器的网络协议栈就会收到这个 ARP 请求，从而将 172.17.0.3 所对应的 MAC 地址回复给 nginx-1 容器。
​        有了这个目的 MAC 地址，nginx-1 容器的 eth0 网卡就可以将数据包发出去。
​        被限制在 Network Namespace 里的容器进程，现实上是通过 Veth Pair 设备 + 宿主机网桥的方式，实现了跟同其他容器的数据互换。
  1. 当一个容器试图连接到另外一个宿主机时，比如：ping 10.168.0.3，它发出的请求数据包，首先经过 docker0 网桥出现在宿主机上。然后根据宿主机的路由表里的直连路由规则（10.168.0.0/24 via eth0)），对 10.168.0.3 的访问请求就会交给宿主机的 eth0 处理。
复制代码
​        这个数据包就会经宿主机的 eth0 网卡转发到宿主机网络上，终极到达 10.168.0.3 对应的宿主机上。固然，这个过程的实现要求这两台宿主机本身是连通的

​        当你遇到容器连不通“外网”的时候，你都应该先试试 docker0 网桥能不能 ping 通，然后检察一下跟 docker0 和 Veth Pair 设备相关的 iptables 规则是不是有非常，往往就可以或许找到问题的答案了。
veth pair： 虚拟1 - docker0 - 虚拟2，每个上面都有一个地址，虚拟1，2不需要解析包，网桥docker0来解析，并做转发操作
“跨主通信”问题

​        假如在另外一台宿主机（比如：10.168.0.3）上，也有一个 Docker 容器。那么，我们的 nginx-1 容器又该怎样访问它呢？
​        在 Docker 的默认设置下，一台宿主机上的 docker0 网桥，和其他宿主机上的 docker0 网桥，没有任何关联，它们相互之间也没办法连通。以是，连接在这些网桥上的容器，自然也没办法进行通信了。
​        假如我们通过软件的方式，创建一个整个集群“公用”的网桥，然后把集群里的所有容器都连接到这个网桥上，不就可以相互通信了吗？

​        构建这种容器网络的核心在于：我们需要在已有的宿主机网络上，再通过软件构建一个覆盖在已有宿主机网络之上的、可以把所有容器连通在一起的虚拟网络。以是，这种技能就被称为：Overlay Network（覆盖网络）。
​        Overlay Network 本身，可以由每台宿主机上的一个“特别网桥”共同构成。比如，当 Node 1 上的 Container 1 要访问 Node 2 上的 Container 3 的时候，Node 1 上的“特别网桥”在收到数据包之后，可以或许通过某种方式，把数据包发送到精确的宿主机，比如 Node 2 上。而 Node 2 上的“特别网桥”在收到数据包后，也可以或许通过某种方式，把数据包转发给精确的容器，比如 Container 3。
​        甚至，每台宿主机上，都不需要有一个这种特别的网桥，而仅仅通过某种方式设置宿主机的路由表，就可以或许把数据包转发到精确的宿主机上。
​        这里的关键在于，容器要想跟外界进行通信，它发出的 IP 包就必须从它的 Network Namespace 里出来，来到宿主机上。而解决这个问题的方法就是：为容器创建一个一端在容器里充当默认网卡、另一端在宿主机上的 Veth Pair 设备。
回复

使用道具 举报

返回列表 发新帖
高级模式
B Color Image Link Quote Code Smilies
您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

登录后关闭弹窗

登录参与点评抽奖  加入IT实名职场社区
去登录

QQ|手机版|qidao123.com IT社区;IT企服评测▪应用市场 ( 浙ICP备20004199|浙ICP备20004199号 )|网站地图

GMT+8, 2026-11-14 08:54 , Processed in 0.035298 second(s), 6 queries , Redis On.

Powered by Discuz! X3.5 Licensed

© 2001-2026 Discuz! Team.

快速回复 返回顶部 返回列表