После настройки прослушивателя очереди для обработки загруженных видео с помощью FFMPEG я несколько раз возвращался на сервер и обнаруживал, что MySql перестал работать. Я проверил место на диске, и оно занято примерно 77% (43 ГБ из 60 ГБ).
Вот мой код на случай, если он пригодится:
public function fire($job, $data)
{
$data = json_decode($data['transcoding_message'], true);
$output_directory = '/home/ubuntu/transcodes/';
$amazon_array = array();
$s3 = AWS::get('s3');
//execute main transcoding thread
$cmd = 'sudo ffmpeg -i ' . $data['temp_file_url'] . ' -y -vcodec libx264 -tune zerolatency -movflags faststart -crf 20 -profile:v main -level:v 3.1 -acodec libfdk_aac -b:a 256k ' . $output_directory. $data['temp_file_key'] . '_HQ.mp4 -vcodec libx264 -s ' . $sq_width . 'x' . $sq_height . ' -tune zerolatency -movflags faststart -crf 25 -profile:v main -level:v 3.1 -acodec libfdk_aac -b:a 256k ' . $output_directory. $data['temp_file_key'] . '_SQ.mp4 -ss ' . $seek_half . ' -f image2 -vf scale=iw/2:-1 -vframes 1 ' . $output_directory. $data['temp_file_key'] . '_thumb.jpg';
exec($cmd." 2>&1", $out, $ret);
if ($ret)
{
Log::error($cmd);
echo 'Processing error' . PHP_EOL;
//there was a problem
return;
}
else
{
//setup file urls
echo 'about to move files';
$hq_url = $this->bucket_root . $data['user_id'] . '/' . $data['temp_file_key'] . '_HQ.mp4';
$sq_url = $this->bucket_root . $data['user_id'] . '/' . $data['temp_file_key'] . '_SQ.mp4';;
$thumb_url = $this->bucket_root . $data['user_id'] . '/' . $data['temp_file_key'] . '_thumb.jpg';
$amazon_array['video_hq_url'] = $data['temp_file_key'] . '_HQ.mp4';
$amazon_array['video_sq_url'] = $data['temp_file_key'] . '_SQ.mp4';
$amazon_array['video_thumb_url'] = $data['temp_file_key'] . '_thumb.jpg';
//copy from temp to permanent
foreach ($amazon_array as $k => $f)
{
$uploader = UploadBuilder::newInstance()
->setClient($s3)
->setSource($output_directory.$f)
->setBucket($this->bucket)
->setKey('users/' . $data['user_id'] . '/' . $f)
->setConcurrency(10)
->setOption('ACL', 'public-read')
->build();
$uploader->getEventDispatcher()->addListener(
'multipart_upload.after_part_upload',
function($event) use ($f) {
// Do whatever you want
}
);
try {
$uploader->upload();
echo "{$k} => Upload complete.\n" . PHP_EOL;
DB::table('items')->where('id', $data['item_id'])->update(array($k => $this->bucket_root. $data['user_id'] . '/' .$f, 'deleted_at' => NULL));
//delete local
unlink($output_directory.$f);
unset($uploader);
} catch (MultipartUploadException $e) {
$uploader->abort();
echo "{$k} => Upload failed.\n" . PHP_EOL;
continue;
}
}
//write to database
DB::table('archives_items')->where('id', $data['archive_item_id'])->update(array('deleted_at' => NULL));
DB::connection('mysql3')->table('video_processing')->where('id', $data['id'])->update(array('finished_processing' => 1));
//delete files
//delete s3
$s3->deleteObject(
array(
'Bucket' => $this->temp_bucket,
'Key' => $data['file_name']
)
);
echo $data['temp_file_url'] . '=>' . " deleted from temp bucket.\n" . PHP_EOL;
DB::connection('mysql3')->table('video_processing')->where('id', $data['id'])->update(array('deleted_at' => \Carbon\Carbon::now()));
}
$job->delete();
// end of processing uploaded video
}
else
{
return;
}
Любые идеи относительно того, почему MySql так умрет?
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (111)
Я хочу упомянуть, что команда php artisan queue: listen запускается через Supervisor и что у меня работает 4 параллельных процесса.
Журнал ошибок:
140613 5:37:52 [Warning] Using unique option prefix myisam-recover instead of myisam-recover-options is deprecated and will be removed in a future release. Please use the full name instead.
140613 5:37:52 [Note] Plugin 'FEDERATED' is disabled.
140613 5:37:52 InnoDB: The InnoDB memory heap is disabled
140613 5:37:52 InnoDB: Mutexes and rw_locks use GCC atomic builtins
140613 5:37:52 InnoDB: Compressed tables use zlib 1.2.3.4
140613 5:37:52 InnoDB: Initializing buffer pool, size = 128.0M
140613 5:37:52 InnoDB: Completed initialization of buffer pool
140613 5:37:52 InnoDB: highest supported file format is Barracuda.
InnoDB: The log sequence number in ibdata files does not match
InnoDB: the log sequence number in the ib_logfiles!
140613 5:37:52 InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
140613 5:37:53 InnoDB: Waiting for the background threads to start
140613 5:37:54 InnoDB: 5.5.37 started; log sequence number 2922980986
140613 5:37:54 [Note] Server hostname (bind-address): '127.0.0.1'; port: 3306
140613 5:37:54 [Note] - '127.0.0.1' resolves to '127.0.0.1';
140613 5:37:54 [Note] Server socket created on IP: '127.0.0.1'.
140613 5:37:55 [Note] Event Scheduler: Loaded 0 events
140613 5:37:55 [Note] /usr/sbin/mysqld: ready for connections.
Version: '5.5.37-0ubuntu0.12.04.1' socket: '/var/run/mysqld/mysqld.sock' port: 3306 (Ubuntu)
140613 5:38:10 [Warning] Using unique option prefix myisam-recover instead of myisam-recover-options is deprecated and will be removed in a future release. Please use the full name instead.
140613 5:38:10 [Note] Plugin 'FEDERATED' is disabled.
140613 5:38:10 InnoDB: The InnoDB memory heap is disabled
140613 5:38:10 InnoDB: Mutexes and rw_locks use GCC atomic builtins
140613 5:38:10 InnoDB: Compressed tables use zlib 1.2.3.4
140613 5:38:10 InnoDB: Initializing buffer pool, size = 128.0M
InnoDB: mmap(137363456 bytes) failed; errno 12
140613 5:38:10 InnoDB: Completed initialization of buffer pool
140613 5:38:10 InnoDB: Fatal error: cannot allocate memory for the buffer pool
140613 5:38:10 [ERROR] Plugin 'InnoDB' init function returned error.
140613 5:38:10 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
140613 5:38:10 [ERROR] Unknown/unsupported storage engine: InnoDB
140613 5:38:10 [ERROR] Aborting
140613 5:38:10 [Note] /usr/sbin/mysqld: Shutdown complete
InnoDB: ошибка mmap (137363456 байт); ошибка 12
140613 5:38:10 InnoDB: Неустранимая ошибка: невозможно выделить память для буферного пула
В вашей системе не хватает памяти. Ошибка 12 - это сигнал ядра ENOMEM.