android - android-位图创建中的优化

我正在编写一个应用程序,该应用程序将从远程TCP连接接收一个图片序列,流由以PGM格式编码的单帧组成,并以9Hz发送,使用后台服务(将完全解码的位图发送到我的MainActivity)可以很容易地处理这样的低帧速率。


 public void run() {


 InetAddress serverAddr = null;



 try {


 serverAddr = InetAddress.getByName(VIDEO_SERVER_ADDR);


 } catch (UnknownHostException e) {


 Log.e(getClass().getName(), e.getMessage());


 e.printStackTrace();


 return;


 }



 Socket socket = null;


 BufferedReader reader = null;



 do {


 try {


 socket = new Socket(serverAddr, VIDEO_SERVER_PORT);



 reader = new BufferedReader(new InputStreamReader(socket.getInputStream()));



 boolean frameStart = false;



 LinkedList<String> frameList = new LinkedList<>();



 while (keepRunning) {


 final String message = reader.readLine();



 if (!frameStart && message.startsWith("F"))


 frameStart = true;


 else if (frameStart && message.startsWith("EF")) {


 frameStart = false;



 final Bitmap bitmap = Bitmap.createBitmap(IR_FRAME_WIDTH, IR_FRAME_HEIGHT, Bitmap.Config.ARGB_8888);


 final Canvas canvas = new Canvas(bitmap);



 final String[] data = frameList.toArray(new String[frameList.size()]);



 canvas.drawBitmap(bitmap, 0, 0, null);



 //Log.d(this.getClass().getName(),"IR FRAME COLLECTED");



 if ((data.length - 6) == IR_FRAME_HEIGHT) {


 float grayScaleRatio = Float.parseFloat(data[2].trim()) / 255.0f;



 for (int y = 0; y < IR_FRAME_HEIGHT; y++) {


 final String line = data[y + 3];


 final String[] points = line.split("s+");



 if (points.length == IR_FRAME_WIDTH) {


 for (int x = 0; x < IR_FRAME_WIDTH; x++) {


 final float grayLevel = Float.parseFloat(points[x]) / grayScaleRatio;



 Paint paint = new Paint();



 paint.setStyle(Paint.Style.FILL);



 final int level = (int)grayLevel;



 paint.setColor(Color.rgb(level, level, level));



 canvas.drawPoint(x, y, paint);


 }


 } else


 Log.d(this.getClass().getName(),"Malformed line");


 }



 final Intent messageIntent = new Intent();



 messageIntent.setAction(VIDEO_BROADCAST_KEY);



 ByteArrayOutputStream stream = new ByteArrayOutputStream();



 bitmap.compress(Bitmap.CompressFormat.PNG, 100, stream);


 bitmap.recycle();


 messageIntent.putExtra(VIDEO_MESSAGE_KEY, stream.toByteArray());


 stream.close();


 sendBroadcast(messageIntent);


 } else


 Log.d(this.getClass().getName(),"Malformed data");



 frameList.clear();


 } else if (frameStart)


 frameList.add(message);


 }



 Thread.sleep(VIDEO_SERVER_RESPAWN);



 } catch (Throwable e) {


 Log.e(getClass().getName(), e.getMessage());


 e.printStackTrace();


 }


 } while (keepRunning);



 if (socket != null) {


 try {


 socket.close();


 } catch (Throwable e) {


 Log.e(getClass().getName(), e.getMessage());


 e.printStackTrace();


 }


 }


 }



message是来自以下文本的一行:


F


P2


160 120


1226


193 141 158 152 193 186 171 177 186 160 195 182 ... (160 times)


 .


 . (120 lines)


 .


278 248 253 261 257 284 310 304 304 272 227 208 ... (160 times)



EF



在MainActivity中,代码处理:


class VideoReceiver extends BroadcastReceiver {


 final public Queue<Bitmap> imagesQueue = new LinkedList<>();



 @Override


 public void onReceive(Context context, Intent intent) {



 try {


 //Log.d(getClass().getName(),"onReceive() called");



 final byte[] data = intent.getByteArrayExtra(VideoService.VIDEO_MESSAGE_KEY);



 final Bitmap bitmap = BitmapFactory.decodeByteArray(data,0,data.length);



 imagesQueue.add(bitmap);



 runOnUiThread(updateVideoTask);


 } catch (Exception ex) {


 ex.printStackTrace();


 }


 }


}



updateVideoTask任务的定义如下:


 updateVideoTask = new Runnable() {


 public void run() {


 if (videoReceiver == null) return;



 if (!videoReceiver.imagesQueue.isEmpty())


 {


 final Bitmap image = videoReceiver.imagesQueue.poll();



 if (image == null) return;



 videoView.setImageBitmap(image);



 Log.d(this.getClass().getName(),"Images to spool:" + videoReceiver.imagesQueue.size());


 }


 }


 };



不幸的是,当我运行该应用程序时,我发现帧速率非常低,并且延迟很大,我从com2获得的唯一提示是:


2019-05-20 16:37:08.817 29566-29580/it.tux.gcs I/art: Background sticky concurrent mark sweep GC freed 88152(3MB) AllocSpace objects, 3(52KB) LOS objects, 22% free, 7MB/10MB, paused 3.937ms total 111.782ms


2019-05-20 16:37:08.832 29566-29587/it.tux.gcs D/skia: Encode PNG Singlethread : 13003 us, width=160, height=120



经过一番检查,我发现:


Paint paint = new Paint();


paint.setStyle(Paint.Style.FILL);



从嵌套循环中不能频繁地调用GC,我找到了另一个主要延迟来源:


final String[] points = line.split("s+");



每次都会消失2ms,所以我决定用更加聪明更快的方法:


final String[] points = line.split("");



不过还是不够,


canvas.drawBitmap(bitmap, 0, 0, null);




sendBroadcast(messageIntent);



仍消耗超过200ms ,我要如何能做得更好?

Activity :


public class MainActivity extends AppCompatActivity implements FrameReadyCallBack {


 private Intent videoServiceIntent;


 private VideoService videoService;


 private boolean bound = false;


 private ImageView surfaceView_video = null;


 private String videoPort ="5002";


 private String videoServerAddr ="192.168.10.107";


 private ServiceConnection serviceConnection = null;



 @Override


 protected void onCreate(Bundle savedInstanceState) {


 super.onCreate(savedInstanceState);


 setContentView(R.layout.activity_main);



 surfaceView_video = findViewById(R.id.surfaceView_video);



 serviceConnection = new ServiceConnection() {


 @Override


 public void onServiceConnected(ComponentName className, IBinder service) {


 VideoService.ImagesCollectorBinder binder = (VideoService.ImagesCollectorBinder) service;


 videoService = binder.getService();


 bound = true;


 videoService.registerCallBack(MainActivity.this); // register


 }



 @Override


 public void onServiceDisconnected(ComponentName arg0) {


 bound = false;


 }


 };



 startVideoService();


 }



 @Override


 public void frameReady(byte[] image_data) {


 //TODO: create image and update surfaceView_video


 }



 public void startVideoService()


 {


 videoServiceIntent = new Intent(this, VideoService.class);



 videoServiceIntent.putExtra(VideoService.LOCAL_PORT_KEY, videoPort);


 videoServiceIntent.putExtra(VideoService.LOCAL_VIDEOSERVER_ADDR_KEY, videoServerAddr);



 startService(videoServiceIntent);


 }



 @Override


 protected void onStart() {


 super.onStart();


 bindService();


 }



 @Override


 protected void onStop() {


 super.onStop();


 unbindService();


 }



 private void bindService() {


 bindService(videoServiceIntent, serviceConnection, Context.BIND_AUTO_CREATE);


 }



 private void unbindService(){


 if (bound) {


 videoService.registerCallBack(null); // unregister


 unbindService(serviceConnection);


 bound = false;


 }


 }


}



服务端:


public class VideoService extends Service {


 public static final String LOCAL_PORT_KEY ="video_port";


 public static final String LOCAL_VIDEOSERVER_ADDR_KEY ="video_server_addr";


 private static final int DEFAULT_VIDEO_PORT = 5002;


 private static final int VIDEO_SERVER_RESPAWN = 2000;



 private volatile FrameReadyCallBack frameReadyCallBack = null;


 private VideoReceiver videoReceiver = null;


 private IBinder videoServiceBinder = new VideoServiceBinder();



 @Nullable


 @Override


 public IBinder onBind(Intent intent) {


 return videoServiceBinder ;


 }



 @Override


 public boolean onUnbind(Intent intent) {


 videoReceiver.kill();


 return super.onUnbind(intent);


 }



 @Override


 public int onStartCommand(Intent intent, int flags, int startId) {


 final int localVideoPort = intent.getIntExtra(LOCAL_PORT_KEY, DEFAULT_VIDEO_PORT);


 final String videoServerAddr = intent.getStringExtra(LOCAL_VIDEOSERVER_ADDR_KEY);



 videoReceiver = new VideoReceiver(videoServerAddr, localVideoPort);


 videoReceiver.start();



 return Service.START_NOT_STICKY;


 }



 public void registerCallBack(FrameReadyCallBack frameReadyCallBack) {


 this.frameReadyCallBack = frameReadyCallBack;


 }



 public class VideoServiceBinder extends Binder {



 public VideoService getService() {


 return VideoService.this;


 }


 }



 private class VideoReceiver extends Thread {


 private boolean keepRunning = true;


 private int VIDEO_SERVER_PORT;


 private String VIDEO_SERVER_ADDR;


 private int bad_frames;


 private int frames;


 private int link_respawn;


 private FrameDecodingStatus status;



 public VideoReceiver(String addr, int listen_port) {


 VIDEO_SERVER_PORT = listen_port;


 VIDEO_SERVER_ADDR = addr;


 }



 public void run() {


 InetAddress serverAddr;


 link_respawn = 0;



 try {


 serverAddr = InetAddress.getByName(VIDEO_SERVER_ADDR);


 } catch (UnknownHostException e) {


 Log.e(getClass().getName(), e.getMessage());


 e.printStackTrace();


 return;


 }



 Socket socket = null;


 DataInputStream stream;



 do {


 bad_frames = 0;


 frames = 0;


 status = FrameDecodingStatus.Idle;



 try {


 socket = new Socket(serverAddr, VIDEO_SERVER_PORT);



 stream = new DataInputStream(new BufferedInputStream(socket.getInputStream()));



 final byte[] _data = new byte[PACKET_SIZE];


 final byte[] _image_data = new byte[IMAGE_SIZE];


 int _data_index = 0;



 while (keepRunning) {


 if (stream.read(_data, 0, _data.length) == 0)


 continue;



 for (byte _byte : _data) {


 if (status == FrameDecodingStatus.Idle) {


 //Wait SoM


 } else if (status == FrameDecodingStatus.Data) {


 //Collect data


 } else {


 frameReadyCallBack.frameReady(_image_data);


 status = FrameDecodingStatus.Idle;


 }


 }


 }



 link_respawn++;


 Thread.sleep(VIDEO_SERVER_RESPAWN);


 Log.d(getClass().getName(),"Link respawn:" + link_respawn);


 } catch (Throwable e) {


 Log.e(getClass().getName(), e.getMessage());


 e.printStackTrace();


 }


 } while (keepRunning);



 if (socket != null) {


 try {


 socket.close();


 } catch (Throwable e) {


 Log.e(getClass().getName(), e.getMessage());


 e.printStackTrace();


 }


 }


 }



 public void kill() {


 keepRunning = false;


 }


 }


}



时间:

首先,通过BroadcastReceiver只提交新图像更改的结果,可以显着提高整体速度,但可以消除这种逻辑。通过Service替换与的通信绑定特性


 // Bind to LocalService


 Intent intent = new Intent(this, LocalService.class);


 bindService(intent, connection, Context.BIND_AUTO_CREATE);



然后接收连接。


/** Defines callbacks for service binding, passed to bindService() */


private ServiceConnection connection = new ServiceConnection() {



 @Override


 public void onServiceConnected(ComponentName className,


 IBinder service) {


 // We've bound to LocalService, cast the IBinder and get LocalService instance


 LocalBinder binder = (LocalBinder) service;


 mService = binder.getService();


 mBound = true;


 }



 @Override


 public void onServiceDisconnected(ComponentName arg0) {


 mBound = false;


 }


};



然后使用Service绑定实例订阅Activity并在Service中使用回调来发布新的数据字节。

...