USBカメラ用パン・チルト台のジョイスティック制御

1.概要

「USBカメラ用パン・チルト台の製作」で作成したUSBカメラのパン・チルト台を、Node-REDのダッシュボードに作成したジョイスティックで制御する方法を説明します。
下の図が、ジョイスティックとカメラ画像を搭載したダッシュボードの画像です。マウスで丸いジョイスティックのドラッグするとその方向にカメラが動きます。マウスのドラッグを止めると中央にジョイスティックは戻り、カメラもパン、チルトの角度0度に戻ります。

2. Node-REDへのジョイスティックの実装

図2-1に、パン・チルト台を制御するNode-REDのフローを示します（Node-REDのフローは本記事の最後のAppendixに載せています）。
大きく4つの部分から成ります。
　1) JoyStick部：node-red-dashboardのui_templateノードを使って、HTML記述でJoyStickを実現しています。
　2) パン・チルト角度情報生成部：JoyStick部から出力される(x,y)座標が(-75,-75)~(75,75)の範囲のデータですので、これを(-90,-90)~(90,90)の角度データに変換します。念のために、 (-75,-75)~(75,75)から外れたデータが来た場合にも範囲内のデータに変換します。
　3) R-MSM制御部：水平方向(パン方向)と垂直方向(チルト方向)のサーボコマンドservo0_deg=,servo1_deg=の形式にtemplateノードを使って変換し、serial outノードを使ってR-MSMに送信します。
　4) Camera画像表示：USBカメラで取得した画像をbase64ノードでbase64形式に変換したデータをLink inノードで受け取り、ダッシュボードに表示します。
次の項から、それぞれの部分に関して説明します。

図2-1. パン・チルト台の制御フロー

2-1. JoyStick部

JoyStick部は、node-red-dashboardのui_templateノードだけです。このノードのtemplate部に、HTML言語でJoySitckを記載しています。
大きくJoyStickの形状の部分(<div>・・・</div>とJoyStickの動作解析と座標を取得する<script>・・・</script>に分かれます。HTMLの詳細はここでは説明しません。ポイントだけ説明します。

2-1-1. JoyStickの形状定義

JoyStickの形状は、以下のHTMLで定義されています。幅と高さが150ピクセル、バックグランド色がピンク、境界色が黒の正方形領域の中に、joytickとして30ピクセルの紺色の円が記述されています。

<div id="joystick-container"
style="width: 150px; height: 150px; position: relative; background-color: pink; border: 2px solid black;">
<div id="joystick" style="
width: 30px;
height: 30px;
background-color: #3f51b5;
border-radius: 50%;
position: absolute;
left: 50%;
top: 50%;
transform: translate(-50%, -50%);
"></div>
</div>

2-1-2. JoyStickの動きの記述

function getPosition(event) {
const rect = container.getBoundingClientRect();
const x = event.clientX – rect.left – rect.width / 2;
const y = event.clientY – rect.top – rect.height / 2;
return { x, y };
}

// スロットリング関数をラップする
const throttledSend = throttle((pos) => {
moveJoystick(pos.x, pos.y);
scope.send({ payload: { x: pos.x, y: pos.y } });
}, 100); // 100ミリ秒ごとに実行

joystick.addEventListener(‘mousedown’, (event) => {
isDragging = true;
const pos = getPosition(event);
moveJoystick(pos.x, pos.y);
});

document.addEventListener(‘mousemove’, (event) => {
if (!isDragging) return;
const pos = getPosition(event);
throttledSend(pos);
});

document.addEventListener(‘mouseup’, () => {
if (isDragging) {
isDragging = false;
moveJoystick(0, 0);
scope.send({ payload: { x: 0, y: 0 } });
}
});

function moveJoystick(x, y) {
joystick.style.transform = `translate(${x}px, ${y}px) translate(-50%, -50%)`;
}
})(scope);

// スロットリング関数の定義
function throttle(func, limit) {
let lastFunc;
let lastRan;
return function(…args) {
const context = this;
if (!lastRan) {
func.apply(context, args);
lastRan = Date.now();
} else {
clearTimeout(lastFunc);
lastFunc = setTimeout(function() {
if ((Date.now() – lastRan) >= limit) {
func.apply(context, args);
lastRan = Date.now();
}
}, limit – (Date.now() – lastRan));
}
};
}
</script>

2-2. パン・チルト角度情報生成部

パン・チルト角度情報生成部は、大きく以下の3つの仕事を行っています。
　1) 送られてきたデータが前回と異なる場合のみ変換する
　2) 座標値が期待の値(-75~75)から外れている場合には、-75もしくは75に矯正する
　3) JoySitck部から送られてきた(x、y)座標を、サーボモータの角度0~180°に変換する
図2-2の①の部分で1)の処理を行い、②の部分で2)と3)の処理を行っています。

図2-2. パン・チルト角度情報生成部のフロー

1)の「送られてきたデータが前回と異なる場合のみ変換する」は、前回の値をflow変数に記録しておき、今回の値がflow変数と異なっているかどうかで判断しています。

2)の「座標値が期待の値(-75~75)から外れている場合には、-75もしくは75に矯正する」と、3) の「JoySitck部から送られてきた(x、y)座標を、サーボモータの角度0~180°に変換する」は、送られたきた値を見て、±75以内であれば、0~180に変換しています。

xとyで変換の仕方が異なります。というのは、-75の時を、0°か180°のどちらに変換するかが異なるからです。joy stickで左に動かしたときにUSBカメラを左向きに、上に動かしたときにUSBカメラを上向きに動かすために、xとyで処理が異なります。

2-3. R-MSM制御部

水平方向(パン方向)と垂直方向(チルト方向)のサーボコマンドservo0_deg=,servo1_deg=の形式にtemplateノードを使って変換し、serial outノード介してR-MSMに送信します。

2-4. Camera画像表示

USBカメラで取得した画像をbase64ノードでbase64形式に変換したデータをLink inノードで受け取り、ダッシュボードに表示します。
HTML言語で、以下のようにimage画像を表示するように、記述します。

<div>
<img src="data:image/png;base64,{{msg.payload}}">
</div>

ここの部分は、PDHにUSBカメラを付けた場合を想定しています。PDHのNode-REDフローの“Setting”タブの一番下のカメラのフローのbase64ノードの出力からLink outノードで送信します。ダッシュボードの「R-MSM GW」タブで、Camera ON/OFFのスイッチを忘れずにONにしてください。デプロイすると自動でOFFになります。

「PDHにつないだUSBカメラ画像(静止画)の表示」など、node-red-contrib-usbcameraを使われている場合には、usbcameraの後ろのbase64ノードを繋ぎその後にui_templateノードを配置してください。「PDH につないだ USB カメラ画像 (静止画 )の表示」を参照して下さい。

Appendix. Node-REDのフロー

ここで紹介したNode-REDのフローを以下に示します。

[{"id":"5d5796a082a7a978","type":"tab","label":"Joy Stick制御","disabled":false,"info":"","env":[]},{"id":"3cb4c84567238249","type":"ui_template","z":"5d5796a082a7a978","group":"ec31a14ff9158ece","name":"JoyStick","order":1,"width":0,"height":0,"format":"<div id=\"joystick-container\"\n style=\"width: 150px; height: 150px; position: relative; background-color: pink; border: 2px solid black;\">\n <div id=\"joystick\" style=\"\n width: 30px;\n height: 30px;\n background-color: #3f51b5;\n border-radius: 50%;\n position: absolute;\n left: 50%;\n top: 50%;\n transform: translate(-50%, -50%);\n \"></div>\n</div>\n\n<script>\n (function(scope) {\n const joystick = document.getElementById('joystick');\n const container = document.getElementById('joystick-container');\n let isDragging = false;\n\n function getPosition(event) {\n const rect = container.getBoundingClientRect();\n const x = event.clientX - rect.left - rect.width / 2;\n const y = event.clientY - rect.top - rect.height / 2;\n return { x, y };\n }\n\n // スロットリング関数をラップする\n const throttledSend = throttle((pos) => {\n moveJoystick(pos.x, pos.y);\n scope.send({ payload: { x: pos.x, y: pos.y } });\n }, 100); // 100ミリ秒ごとに実行\n\n joystick.addEventListener('mousedown', (event) => {\n isDragging = true;\n const pos = getPosition(event);\n moveJoystick(pos.x, pos.y);\n });\n\n document.addEventListener('mousemove', (event) => {\n if (!isDragging) return;\n const pos = getPosition(event);\n throttledSend(pos);\n });\n\n document.addEventListener('mouseup', () => {\n if (isDragging) {\n isDragging = false;\n moveJoystick(0, 0);\n scope.send({ payload: { x: 0, y: 0 } });\n }\n });\n\n function moveJoystick(x, y) {\n joystick.style.transform = `translate(${x}px, ${y}px) translate(-50%, -50%)`;\n }\n})(scope);\n\n// スロットリング関数の定義\nfunction throttle(func, limit) {\n let lastFunc;\n let lastRan;\n return function(...args) {\n const context = this;\n if (!lastRan) {\n func.apply(context, args);\n lastRan = Date.now();\n } else {\n clearTimeout(lastFunc);\n lastFunc = setTimeout(function() {\n if ((Date.now() - lastRan) >= limit) {\n func.apply(context, args);\n lastRan = Date.now();\n }\n }, limit - (Date.now() - lastRan));\n }\n };\n}\n</script>","storeOutMessages":true,"fwdInMessages":true,"resendOnRefresh":true,"templateScope":"local","className":"","x":100,"y":160,"wires":[["618946e0489d5251","9c99e02f3a8ce2f9"]]},{"id":"618946e0489d5251","type":"debug","z":"5d5796a082a7a978","name":"debug 48","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":240,"y":120,"wires":[]},{"id":"9c99e02f3a8ce2f9","type":"switch","z":"5d5796a082a7a978","name":"x","property":"payload.x","propertyType":"msg","rules":[{"t":"neq","v":"x","vt":"flow"}],"checkall":"true","repair":false,"outputs":1,"x":230,"y":180,"wires":[["c530156fcd6812f5"]]},{"id":"c530156fcd6812f5","type":"switch","z":"5d5796a082a7a978","name":"y","property":"payload.y","propertyType":"msg","rules":[{"t":"neq","v":"y","vt":"flow"}],"checkall":"true","repair":false,"outputs":1,"x":350,"y":180,"wires":[["cdb6da291108b349","cf1fb1789ba01cf4"]]},{"id":"cdb6da291108b349","type":"change","z":"5d5796a082a7a978","name":"set flow","rules":[{"t":"set","p":"x","pt":"flow","to":"payload.x","tot":"msg"},{"t":"set","p":"y","pt":"flow","to":"payload.y","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":480,"y":120,"wires":[[]]},{"id":"cf1fb1789ba01cf4","type":"switch","z":"5d5796a082a7a978","name":"x","property":"payload.x","propertyType":"msg","rules":[{"t":"lte","v":"75","vt":"num"},{"t":"gte","v":"-75","vt":"num"},{"t":"gt","v":"75","vt":"num"},{"t":"lt","v":"-75","vt":"num"}],"checkall":"false","repair":false,"outputs":4,"x":470,"y":180,"wires":[["e537c0c02f630a1d"],["e537c0c02f630a1d"],["76d828ea20e2b744"],["62ac31ae37b0759a"]]},{"id":"76d828ea20e2b744","type":"change","z":"5d5796a082a7a978","name":"> 75","rules":[{"t":"set","p":"payload.x","pt":"msg","to":"180","tot":"num"},{"t":"set","p":"payload","pt":"msg","to":"$abs(payload.x-180)","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":590,"y":200,"wires":[["e7ead1784802122c"]]},{"id":"62ac31ae37b0759a","type":"change","z":"5d5796a082a7a978","name":"< -75","rules":[{"t":"set","p":"payload.x","pt":"msg","to":"0","tot":"num"},{"t":"set","p":"payload","pt":"msg","to":"$abs(payload.x-180)","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":590,"y":240,"wires":[["e7ead1784802122c"]]},{"id":"e537c0c02f630a1d","type":"change","z":"5d5796a082a7a978","name":"≦|75|","rules":[{"t":"set","p":"payload.x","pt":"msg","to":"$floor(payload.x/75*90)+90\t","tot":"jsonata"},{"t":"set","p":"payload.x","pt":"msg","to":"$abs(payload.x-180)\t","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":590,"y":160,"wires":[["e7ead1784802122c"]]},{"id":"e7ead1784802122c","type":"switch","z":"5d5796a082a7a978","name":"y","property":"payload.y","propertyType":"msg","rules":[{"t":"lte","v":"75","vt":"num"},{"t":"gte","v":"-75","vt":"num"},{"t":"gt","v":"75","vt":"num"},{"t":"lt","v":"-75","vt":"num"}],"checkall":"false","repair":false,"outputs":4,"x":730,"y":160,"wires":[["3fa37051ada36db5"],["3fa37051ada36db5"],["f0a96d1f4298d99a"],["1bc86fead87dcca5"]]},{"id":"3fa37051ada36db5","type":"change","z":"5d5796a082a7a978","name":"≦|75|","rules":[{"t":"set","p":"payload.y","pt":"msg","to":"$floor(payload.y/75*90)+90\t","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":850,"y":140,"wires":[["ee0e9b64ad20ad37","76854a109dd9c735"]]},{"id":"f0a96d1f4298d99a","type":"change","z":"5d5796a082a7a978","name":"> 75","rules":[{"t":"set","p":"payload.y","pt":"msg","to":"180","tot":"num"}],"action":"","property":"","from":"","to":"","reg":false,"x":850,"y":180,"wires":[["ee0e9b64ad20ad37","76854a109dd9c735"]]},{"id":"1bc86fead87dcca5","type":"change","z":"5d5796a082a7a978","name":"< -75","rules":[{"t":"set","p":"payload.y","pt":"msg","to":"0","tot":"num"}],"action":"","property":"","from":"","to":"","reg":false,"x":850,"y":220,"wires":[["ee0e9b64ad20ad37","76854a109dd9c735"]]},{"id":"ee0e9b64ad20ad37","type":"template","z":"5d5796a082a7a978","name":"horizontal","field":"payload","fieldType":"msg","format":"handlebars","syntax":"mustache","template":"servo0_deg={{payload.x}}","output":"str","x":1000,"y":160,"wires":[["624a59d52be062a0","5a2d50909ad306af"]]},{"id":"76854a109dd9c735","type":"template","z":"5d5796a082a7a978","name":"vertical","field":"payload","fieldType":"msg","format":"handlebars","syntax":"mustache","template":"servo1_deg={{payload.y}}","output":"str","x":1000,"y":200,"wires":[["624a59d52be062a0","5a2d50909ad306af"]]},{"id":"624a59d52be062a0","type":"debug","z":"5d5796a082a7a978","name":"debug 49","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":1140,"y":160,"wires":[]},{"id":"5a2d50909ad306af","type":"serial out","z":"5d5796a082a7a978","name":"","serial":"21bcbf72cb50012b","x":1160,"y":200,"wires":[]},{"id":"d941badaeaf08d68","type":"ui_template","z":"5d5796a082a7a978","group":"74630e4c8f852449","name":"JPEG表示","order":1,"width":8,"height":6,"format":"<div>\n <img src=\"data:image/png;base64,{{msg.payload}}\">\n</div>","storeOutMessages":true,"fwdInMessages":true,"resendOnRefresh":false,"templateScope":"local","className":"","x":170,"y":320,"wires":[[]]},{"id":"72aaf2a44d162466","type":"link in","z":"5d5796a082a7a978","name":"link in 7","links":["d3392e86c2d2a0f5"],"x":65,"y":320,"wires":[["d941badaeaf08d68"]]},{"id":"ec31a14ff9158ece","type":"ui_group","name":"Joy Stick","tab":"d1b3fd94dff09b4b","order":1,"disp":true,"width":"6","collapse":false,"className":""},{"id":"21bcbf72cb50012b","type":"serial-port","name":"","serialport":"/dev/rfcomm0","serialbaud":"1000000","databits":"8","parity":"none","stopbits":"1","waitfor":"","dtr":"none","rts":"none","cts":"none","dsr":"none","newline":"\\n","bin":"false","out":"char","addchar":"\\n","responsetimeout":"10000"},{"id":"74630e4c8f852449","type":"ui_group","name":"Web Camera","tab":"d1b3fd94dff09b4b","order":1,"disp":true,"width":8,"collapse":false,"className":""},{"id":"d1b3fd94dff09b4b","type":"ui_tab","name":"WebCamera2","icon":"dashboard","disabled":false,"hidden":false}]