I'm focused on Distributed Systems and Physical Design - Placement.
- To read more about me: My website
- To find what I'm learning: Note
- To read my blog: Blog
An easy-to-use Map Reduce Go parallel-computing framework inspired by 2021 6.824 lab1. It supports multiple workers threads on a single machine and multiple processes on a single machine right now.
Home Page: https://pkg.go.dev/github.com/BWbwchen/MapReduce
License: MIT License
I'm focused on Distributed Systems and Physical Design - Placement.
when @kiarash8112 run make test_multi_node
, he get following error
rm output/*
rm: cannot remove 'output/*': No such file or directory
make: [Makefile:10: clean] Error 1 (ignored)
rm cmd/*.so
rm: cannot remove 'cmd/*.so': No such file or directory
make: [Makefile:11: clean] Error 1 (ignored)
go build -race -buildmode=plugin -o cmd/wc.so ./mrapps/wc.go
go build -race -buildmode=plugin -o cmd/merge.so ./mrapps/merge.go
rm output/*
rm: cannot remove 'output/*': No such file or directory
make: [Makefile:27: test_multi_node] Error 1 (ignored)
rm mr-out-*
rm: cannot remove 'mr-out-*': No such file or directory
make: [Makefile:28: test_multi_node] Error 1 (ignored)
go run -race cmd/master.go -i 'txt/*' -p 'cmd/wc.so' -r 4 -w 8 &
sleep 1
go run -race cmd/worker.go -i 'txt/*' -p 'cmd/wc.so' -r 4 -w 1 &
go run -race cmd/worker.go -i 'txt/*' -p 'cmd/wc.so' -r 4 -w 2 &
go run -race cmd/worker.go -i 'txt/*' -p 'cmd/wc.so' -r 4 -w 3 &
go run -race cmd/worker.go -i 'txt/*' -p 'cmd/wc.so' -r 4 -w 4 &
go run -race cmd/worker.go -i 'txt/*' -p 'cmd/wc.so' -r 4 -w 5 &
go run -race cmd/worker.go -i 'txt/*' -p 'cmd/wc.so' -r 4 -w 6 &
go run -race cmd/worker.go -i 'txt/*' -p 'cmd/wc.so' -r 4 -w 7 &
go run -race cmd/worker.go -i 'txt/*' -p 'cmd/wc.so' -r 4 -w 8
INFO[0000] [Master] Master gRPC server start
TRAC[0000] [Master] Wait for enough workers
INFO[0000] Worker gRPC server start
INFO[0000] Worker load plugin finish
TRAC[0000] With Time out
TRAC[0000] Start RPC call
INFO[0000] [Master] Worker register success
TRAC[0000] End RPC call
INFO[0000] Worker register itself finish
INFO[0000] Worker gRPC server start
INFO[0000] Worker load plugin finish
TRAC[0000] With Time out
TRAC[0000] Start RPC call
INFO[0001] [Master] Worker register success
TRAC[0000] End RPC call
INFO[0000] Worker register itself finish
INFO[0000] Worker gRPC server start
INFO[0000] Worker gRPC server start
INFO[0000] Worker load plugin finish
TRAC[0000] With Time out
TRAC[0000] Start RPC call
INFO[0001] [Master] Worker register success
TRAC[0000] End RPC call
INFO[0000] Worker register itself finish
INFO[0000] Worker load plugin finish
TRAC[0000] With Time out
TRAC[0000] Start RPC call
INFO[0001] [Master] Worker register success
TRAC[0000] End RPC call
INFO[0000] Worker register itself finish
INFO[0000] Worker gRPC server start
INFO[0000] Worker load plugin finish
TRAC[0000] With Time out
TRAC[0000] Start RPC call
INFO[0001] [Master] Worker register success
TRAC[0000] End RPC call
INFO[0000] Worker register itself finish
INFO[0000] Worker gRPC server start
INFO[0000] Worker gRPC server start
INFO[0000] Worker load plugin finish
TRAC[0000] With Time out
TRAC[0000] Start RPC call
INFO[0001] [Master] Worker register success
TRAC[0000] End RPC call
INFO[0000] Worker register itself finish
INFO[0000] Worker load plugin finish
TRAC[0000] With Time out
TRAC[0000] Start RPC call
INFO[0001] [Master] Worker register success
TRAC[0000] End RPC call
INFO[0000] Worker register itself finish
INFO[0000] Worker gRPC server start
INFO[0000] Worker load plugin finish
TRAC[0000] With Time out
TRAC[0000] Start RPC call
TRAC[0001] [Master] Enough workers!
INFO[0001] [Master] Worker register success
TRAC[0001] [Master] Start distribute workload
TRAC[0001] Add map input file
TRAC[0001] Add map input file
TRAC[0001] Add map input file
TRAC[0001] Add map input file
TRAC[0000] End RPC call
INFO[0000] Worker register itself finish
TRAC[0001] Add map input file
TRAC[0001] Add map input file
TRAC[0001] Add map input file
TRAC[0001] Add map input file
TRAC[0001] [Master] End distribute workload
TRAC[0001] [Master] Start Map task
INFO[0001] [Master] Finding available workers to execute 8/8
INFO[0001] [Worker] Start Map
INFO[0000] [Worker] Start Map
TRAC[0000] [Worker] Start Mapping
TRAC[0000] [Worker] Finish Mapping
TRAC[0001] [Worker] Start Mapping
TRAC[0001] [Worker] Finish Mapping
TRAC[0001] [Worker] Start partition intermediate kv
INFO[0000] [Worker] Start Map
TRAC[0000] [Worker] Start Mapping
TRAC[0000] [Worker] Start partition intermediate kv
TRAC[0000] [Worker] End partition intermediate kv
TRAC[0001] [Worker] End partition intermediate kv
TRAC[0000] [Worker] Write intermediate kv to file
TRAC[0000] [Worker] Finish Mapping
INFO[0000] [Worker] Start Map
TRAC[0000] [Worker] Start Mapping
TRAC[0000] [Worker] End Write intermediate kv to file
TRAC[0000] [Worker] Start partition intermediate kv
TRAC[0000] [Worker] End partition intermediate kv
TRAC[0000] [Worker] Tell Master the intermediate info
TRAC[0000] [Worker] Finish Mapping
TRAC[0000] [Worker] Write intermediate kv to file
TRAC[0000] [Worker] Start partition intermediate kv
TRAC[0000] [Worker] End partition intermediate kv
TRAC[0000] [Worker] Write intermediate kv to file
INFO[0000] [Worker] Start Map
TRAC[0000] [Worker] Start Mapping
TRAC[0000] [Worker] Finish Mapping
TRAC[0000] [Worker] Start partition intermediate kv
TRAC[0001] [Worker] Write intermediate kv to file
TRAC[0000] [Worker] End partition intermediate kv
TRAC[0000] [Worker] Write intermediate kv to file
TRAC[0000] [Worker] End Write intermediate kv to file
TRAC[0000] [Worker] Tell Master the intermediate info
TRAC[0000] [Worker] End Write intermediate kv to file
TRAC[0000] [Worker] Tell Master the intermediate info
TRAC[0000] [Worker] End Write intermediate kv to file
TRAC[0000] [Worker] Tell Master the intermediate info
TRAC[0001] [Worker] End Write intermediate kv to file
TRAC[0001] [Worker] Tell Master the intermediate info
INFO[0001] [Master] dfbe266c-095f-43c0-ba22-f03c1a3192dc update IMD info success
TRAC[0000] [Worker] Finish Tell Master the intermediate info
INFO[0000] [Worker] Finish Map Task
INFO[0001] [Master] 52800c38-289e-40ac-98c8-e95b01ba7f8e update IMD info success
INFO[0001] [Master] 4744d331-a214-4015-abbb-f32e2239dd5d update IMD info success
INFO[0001] [Master] 84a20685-69ef-4b76-b41d-046ab8fa17ea update IMD info success
INFO[0001] [Master] 8be49474-e777-48ee-a1cb-4ca5f5b88c30 update IMD info success
TRAC[0000] [Worker] Finish Tell Master the intermediate info
INFO[0000] [Worker] Finish Map Task
INFO[0000] [Worker] Start Map
TRAC[0000] [Worker] Start Mapping
TRAC[0000] [Worker] Finish Mapping
TRAC[0000] [Worker] Start partition intermediate kv
TRAC[0000] [Worker] End partition intermediate kv
TRAC[0000] [Worker] Write intermediate kv to file
TRAC[0000] [Worker] Finish Tell Master the intermediate info
INFO[0000] [Worker] Finish Map Task
INFO[0000] [Worker] Start Map
TRAC[0000] [Worker] Start Mapping
TRAC[0000] [Worker] Finish Mapping
TRAC[0000] [Worker] Start partition intermediate kv
TRAC[0001] [Worker] Finish Tell Master the intermediate info
INFO[0001] [Worker] Finish Map Task
TRAC[0000] [Worker] End partition intermediate kv
TRAC[0000] [Worker] Write intermediate kv to file
TRAC[0000] [Worker] End Write intermediate kv to file
TRAC[0000] [Worker] Tell Master the intermediate info
INFO[0000] [Worker] Start Map
TRAC[0000] [Worker] Start Mapping
TRAC[0000] [Worker] End Write intermediate kv to file
INFO[0001] [Master] 09a01a3b-2239-4c63-9428-17bad0ea82f9 update IMD info success
TRAC[0000] [Worker] Finish Mapping
TRAC[0000] [Worker] Start partition intermediate kv
TRAC[0000] [Worker] End partition intermediate kv
TRAC[0000] [Worker] Write intermediate kv to file
TRAC[0000] [Worker] End Write intermediate kv to file
TRAC[0000] [Worker] Tell Master the intermediate info
TRAC[0000] [Worker] Tell Master the intermediate info
TRAC[0000] [Worker] Finish Tell Master the intermediate info
INFO[0000] [Worker] Finish Map Task
INFO[0001] [Master] 07e24826-d4d9-44dd-98ed-0e704c6e2a85 update IMD info success
TRAC[0000] [Worker] Finish Tell Master the intermediate info
INFO[0001] [Master] 40d0408b-cacd-41c1-b79d-fec6a7de044f update IMD info success
INFO[0000] [Worker] Finish Map Task
TRAC[0000] [Worker] Finish Tell Master the intermediate info
INFO[0000] [Worker] Finish Map Task
TRAC[0000] [Worker] Finish Tell Master the intermediate info
INFO[0000] [Worker] Finish Map Task
TRAC[0001] [Master] End Map task
TRAC[0001] [Master] Start Reduce task
INFO[0001] [Master] Finding available workers to execute 4/8
INFO[0001] [Worker] Start Reduce
TRAC[0001] [Worker] Get intermediate file
INFO[0000] [Worker] Start Reduce
TRAC[0000] [Worker] Get intermediate file
INFO[0000] [Worker] Start Reduce
TRAC[0000] [Worker] Get intermediate file
INFO[0000] [Worker] Start Reduce
TRAC[0000] [Worker] Get intermediate file
INFO[0000] [Worker] RPC Get intermediate file
INFO[0000] [Worker] RPC Get intermediate file
INFO[0000] [Worker] RPC Get intermediate file
INFO[0000] [Worker] RPC Get intermediate file
INFO[0001] [Worker] RPC Get intermediate file
INFO[0001] [Worker] RPC Get intermediate file
INFO[0001] [Worker] RPC Get intermediate file
INFO[0001] [Worker] RPC Get intermediate file
INFO[0000] [Worker] RPC Get intermediate file
INFO[0000] [Worker] RPC Get intermediate file
INFO[0000] [Worker] RPC Get intermediate file
INFO[0000] [Worker] RPC Get intermediate file
INFO[0000] [Worker] RPC Get intermediate file
INFO[0000] [Worker] RPC Get intermediate file
INFO[0000] [Worker] RPC Get intermediate file
INFO[0000] [Worker] RPC Get intermediate file
INFO[0000] [Worker] RPC Get intermediate file
INFO[0000] [Worker] RPC Get intermediate file
INFO[0000] [Worker] RPC Get intermediate file
INFO[0000] [Worker] RPC Get intermediate file
INFO[0000] [Worker] RPC Get intermediate file
INFO[0000] [Worker] RPC Get intermediate file
INFO[0000] [Worker] RPC Get intermediate file
INFO[0000] [Worker] RPC Get intermediate file
INFO[0000] [Worker] RPC Get intermediate file
INFO[0000] [Worker] RPC Get intermediate file
TRAC[0001] [Worker] Sort intermediate KV
TRAC[0001] [Worker] Start Reducing
TRAC[0001] [Worker] End Reducing
INFO[0001] [Worker] End Reduce
INFO[0000] [Worker] RPC Get intermediate file
INFO[0000] [Worker] RPC Get intermediate file
TRAC[0000] [Worker] Sort intermediate KV
TRAC[0000] [Worker] Start Reducing
TRAC[0000] [Worker] End Reducing
INFO[0000] [Worker] End Reduce
INFO[0000] [Worker] RPC Get intermediate file
TRAC[0000] [Worker] Sort intermediate KV
TRAC[0000] [Worker] Start Reducing
TRAC[0000] [Worker] End Reducing
INFO[0000] [Worker] End Reduce
INFO[0000] [Worker] RPC Get intermediate file
INFO[0000] [Worker] RPC Get intermediate file
INFO[0000] [Worker] RPC Get intermediate file
TRAC[0000] [Worker] Sort intermediate KV
TRAC[0000] [Worker] Start Reducing
TRAC[0000] [Worker] End Reducing
INFO[0000] [Worker] End Reduce
TRAC[0001] [Master] End Reduce task
TRAC[0001] [Master] End Workers Start
INFO[0001] [Worker] End worker
INFO[0000] [Worker] End worker
INFO[0000] [Worker] End worker
INFO[0000] [Worker] End worker
INFO[0000] [Worker] End worker
INFO[0000] [Worker] End worker
INFO[0000] [Worker] End worker
INFO[0000] [Worker] End worker
TRAC[0001] [Master] End Workers done
go run -race cmd/master.go -i 'mr-out-*' -p 'cmd/merge.so' -r 1 -w 4 &
sleep 1
INFO[0000] [Master] Master gRPC server start
TRAC[0000] [Master] Wait for enough workers
go run -race cmd/worker.go -i 'mr-out-*' -p 'cmd/merge.so' -r 1 -w 1 &
go run -race cmd/worker.go -i 'mr-out-*' -p 'cmd/merge.so' -r 1 -w 2 &
go run -race cmd/worker.go -i 'mr-out-*' -p 'cmd/merge.so' -r 1 -w 3 &
go run -race cmd/worker.go -i 'mr-out-*' -p 'cmd/merge.so' -r 1 -w 4
INFO[0000] Worker gRPC server start
INFO[0000] Worker load plugin finish
TRAC[0000] With Time out
TRAC[0000] Start RPC call
INFO[0001] [Master] Worker register success
TRAC[0000] End RPC call
INFO[0000] Worker register itself finish
INFO[0000] Worker gRPC server start
INFO[0000] Worker load plugin finish
TRAC[0000] With Time out
TRAC[0000] Start RPC call
INFO[0002] [Master] Worker register success
TRAC[0000] End RPC call
INFO[0000] Worker register itself finish
INFO[0000] Worker gRPC server start
INFO[0000] Worker load plugin finish
TRAC[0000] With Time out
TRAC[0000] Start RPC call
INFO[0002] [Master] Worker register success
TRAC[0000] End RPC call
INFO[0000] Worker register itself finish
INFO[0000] Worker gRPC server start
INFO[0000] Worker load plugin finish
TRAC[0000] With Time out
TRAC[0000] Start RPC call
INFO[0002] [Master] Worker register success
TRAC[0002] [Master] Enough workers!
TRAC[0002] [Master] Start distribute workload
TRAC[0002] Add map input file
TRAC[0002] Add map input file
TRAC[0002] Add map input file
TRAC[0002] Add map input file
TRAC[0002] Add map input file
TRAC[0002] Add map input file
TRAC[0002] Add map input file
TRAC[0002] Add map input file
TRAC[0002] Add map input file
TRAC[0002] Add map input file
TRAC[0002] Add map input file
TRAC[0002] Add map input file
TRAC[0002] Add map input file
TRAC[0002] Add map input file
TRAC[0002] Add map input file
TRAC[0002] Add map input file
TRAC[0002] [Master] End distribute workload
TRAC[0002] [Master] Start Map task
INFO[0002] [Master] Finding available workers to execute 4/4
TRAC[0000] End RPC call
INFO[0000] Worker register itself finish
INFO[0000] [Worker] Start Map
INFO[0000] [Worker] Start Map
TRAC[0000] [Worker] Start Mapping
TRAC[0000] [Worker] Start Mapping
INFO[0000] [Worker] Start Map
TRAC[0000] [Worker] Finish Mapping
TRAC[0000] [Worker] Finish Mapping
TRAC[0000] [Worker] Start partition intermediate kv
TRAC[0000] [Worker] Start partition intermediate kv
TRAC[0000] [Worker] Start Mapping
INFO[0000] [Worker] Start Map
TRAC[0000] [Worker] Start Mapping
TRAC[0000] [Worker] Finish Mapping
TRAC[0000] [Worker] Start partition intermediate kv
TRAC[0000] [Worker] End partition intermediate kv
TRAC[0000] [Worker] Write intermediate kv to file
TRAC[0000] [Worker] End Write intermediate kv to file
TRAC[0000] [Worker] Tell Master the intermediate info
panic: runtime error: index out of range [1] with length 1
goroutine 27 [running]:
plugin/unnamed-8794b001124bf89ba2b7e1417ecabf0b0c406174.Map({0x0?, 0x0?}, {0x0, 0x0}, {0xc0001808a0})
/home/user/Desktop/MapReduce/mrapps/merge.go:20 +0x1b1
github.com/BWbwchen/MapReduce/worker.(*Worker).Map.func1(0xc0001948c0, {0x0, 0x0})
/home/user/Desktop/MapReduce/worker/worker.go:61 +0xc8
created by github.com/BWbwchen/MapReduce/worker.(*Worker).Map in goroutine 35
/home/user/Desktop/MapReduce/worker/worker.gopanic: runtime error: index out of range [1] with length 1
goroutine 10 [running]:
plugin/unnamed-8794b001124bf89ba2b7e1417ecabf0b0c406174.Map({0x0?, 0x0?}, {0x0, 0x0}, {0xc00028e480})
/home/user/Desktop/MapReduce/mrapps/merge.go:20 +0x1b1
github.com/BWbwchen/MapReduce/worker.(*Worker).Map.func1(0xc000282690, {0x0, :600x0})
/home/user/Desktop/MapReduce/worker/worker.go:61 +0xc8
+created by github.com/BWbwchen/MapReduce/worker.(*Worker).Map in goroutine 6
0x185 /home/user/Desktop/MapReduce/worker/worker.go:60 +0x185
INFO[0002] [Master] 9503430e-b944-4d68-8531-1fd401a48b07 update IMD info success
TRAC[0000] [Worker] Finish Tell Master the intermediate info
INFO[0000] [Worker] Finish Map Task
panic: runtime error: index out of range [1] with length 1
goroutine 28 [running]:
plugin/unnamed-8794b001124bf89ba2b7e1417ecabf0b0c406174.Map({0x0?, 0x0?}, {0x0, 0x0}, {0xc000094d20})
/home/user/Desktop/MapReduce/mrapps/merge.go:20 +0x1b1
github.com/BWbwchen/MapReduce/worker.(*Worker).Map.func1(0xc000096eb0, {0x0, 0x0})
/home/user/Desktop/MapReduce/worker/worker.go:61 +0xc8
created by github.com/BWbwchen/MapReduce/worker.(*Worker).Map in goroutine 25
/home/user/Desktop/MapReduce/worker/worker.go:60 +0x185
exit status 2
exit status 2
WARN[0002] [Master]: error reading from server: EOF
WARN[0002] [Master]: error reading from server: EOF
INFO[0002] [Master] Finding available workers to execute 1/4
INFO[0002] [Master] Re-execute Map Task from 9c0bc0cd-5305-49f7-87ec-c60dd346ab94 To 9503430e-b944-4d68-8531-1fd401a48b07
INFO[0002] [Master] Finding available workers to execute 1/4
INFO[0000] [Worker] Start Map
TRAC[0000] [Worker] Start Mapping
TRAC[0000] [Worker] Finish Mapping
TRAC[0000] [Worker] Start partition intermediate kv
make: *** [Makefile:44: test_multi_node] Error 1
panic: runtime error: index out of range [1] with length 1
goroutine 70 [running]:
plugin/unnamed-8794b001124bf89ba2b7e1417ecabf0b0c406174.Map({0x1?, 0x1?}
, {0x0, 0x0}, {0xc000180420})
/home/user/Desktop/MapReduce/mrapps/merge.go:20 +0x1b1
github.com/BWbwchen/MapReduce/worker.(*Worker).Map.func1(0xc000194aa0, {0x0, 0x0})
/home/user/Desktop/MapReduce/worker/worker.go:61 +0xc8
created by github.com/BWbwchen/MapReduce/worker.(*Worker).Map in goroutine 66
/home/user/Desktop/MapReduce/worker/worker.go:60 +0x185
exit status 2
WARN[0002] [Master]: error reading from server: EOF
exit status 2
==================
WARNING: DATA RACE
Write at 0x00c000172170 by goroutine 33:
github.com/BWbwchen/MapReduce/master.(*WorkerInfo).SetState()
/home/user/Desktop/MapReduce/master/workerInfo.go:36 +0x4d
github.com/BWbwchen/MapReduce/master.(*Master).distributeMapTask.func1()
/home/user/Desktop/MapReduce/master/master.go:220 +0x584
github.com/BWbwchen/MapReduce/master.(*Master).distributeMapTask.func4()
/home/user/Desktop/MapReduce/master/master.go:228 +0xa6
Previous read at 0x00c000172170 by goroutine 7:
github.com/BWbwchen/MapReduce/master.(*WorkerInfo).Broken()
/home/user/Desktop/MapReduce/master/workerInfo.go:41 +0x49e
github.com/BWbwchen/MapReduce/master.(*Master).availableWorkers()
/home/user/Desktop/MapReduce/master/master.go:144 +0x450
github.com/BWbwchen/MapReduce/master.(*Master).distributeMapTask()
/home/user/Desktop/MapReduce/master/master.go:233 +0x60e
github.com/BWbwchen/MapReduce/master.StartMaster()
/home/user/Desktop/MapReduce/master/mainInterface.go:47 +0x330
github.com/BWbwchen/MapReduce.startMaster.func1()
/home/user/Desktop/MapReduce/util.go:102 +0x89
Goroutine 33 (running) created at:
github.com/BWbwchen/MapReduce/master.(*Master).distributeMapTask()
/home/user/Desktop/MapReduce/master/master.go:215 +0x512
github.com/BWbwchen/MapReduce/master.StartMaster()
/home/user/Desktop/MapReduce/master/mainInterface.go:47 +0x330
github.com/BWbwchen/MapReduce.startMaster.func1()
/home/user/Desktop/MapReduce/util.go:102 +0x89
Goroutine 7 (running) created at:
github.com/BWbwchen/MapReduce.startMaster()
/home/user/Desktop/MapReduce/util.go:101 +0x2a4
github.com/BWbwchen/MapReduce.StartMaster()
/home/user/Desktop/MapReduce/masterNode.go:4 +0x27
main.main()
/home/user/Desktop/MapReduce/cmd/master.go:8 +0x18
==================
WARN[0002] [Master]: error reading from server: EOF
==================
WARNING: DATA RACE
Write at 0x00c0001720e0 by goroutine 58:
github.com/BWbwchen/MapReduce/master.(*WorkerInfo).SetState()
/home/user/Desktop/MapReduce/master/workerInfo.go:36 +0x4d
github.com/BWbwchen/MapReduce/master.(*Master).distributeMapTask.func2()
/home/user/Desktop/MapReduce/master/master.go:248 +0x5ac
github.com/BWbwchen/MapReduce/master.(*Master).distributeMapTask.func5()
/home/user/Desktop/MapReduce/master/master.go:255 +0x91
Previous read at 0x00c0001720e0 by goroutine 7:
github.com/BWbwchen/MapReduce/master.(*WorkerInfo).Broken()
/home/user/Desktop/MapReduce/master/workerInfo.go:41 +0x49e
github.com/BWbwchen/MapReduce/master.(*Master).availableWorkers()
/home/user/Desktop/MapReduce/master/master.go:144 +0x450
github.com/BWbwchen/MapReduce/master.(*Master).distributeMapTask()
/home/user/Desktop/MapReduce/master/master.go:233 +0x60e
github.com/BWbwchen/MapReduce/master.StartMaster()
/home/user/Desktop/MapReduce/master/mainInterface.go:47 +0x330
github.com/BWbwchen/MapReduce.startMaster.func1()
/home/user/Desktop/MapReduce/util.go:102 +0x89
Goroutine 58 (running) created at:
github.com/BWbwchen/MapReduce/master.(*Master).distributeMapTask()
/home/user/Desktop/MapReduce/master/master.go:242 +0x5b9
github.com/BWbwchen/MapReduce/master.StartMaster()
/home/user/Desktop/MapReduce/master/mainInterface.go:47 +0x330
github.com/BWbwchen/MapReduce.startMaster.func1()
/home/user/Desktop/MapReduce/util.go:102 +0x89
Goroutine 7 (running) created at:
github.com/BWbwchen/MapReduce.startMaster()
/home/user/Desktop/MapReduce/util.go:101 +0x2a4
github.com/BWbwchen/MapReduce.StartMaster()
/home/user/Desktop/MapReduce/masterNode.go:4 +0x27
main.main()
/home/user/Desktop/MapReduce/cmd/master.go:8 +0x18
==================
PANI[0002] No enough worker!0 1
panic: (*logrus.Entry) 0xc0000ac9a0
goroutine 6 [running]:
github.com/sirupsen/logrus.(*Entry).log(0xc0000ac930, 0x0, {0xc00009e120, 0x14})
/home/user/Desktop/MapReduce/vendor/github.com/sirupsen/logrus/entry.go:259 +0x97f
github.com/sirupsen/logrus.(*Entry).Log(0xc0000ac930, 0x0, {0xc00019bb80, 0x3, 0x3})
/home/user/Desktop/MapReduce/vendor/github.com/sirupsen/logrus/entry.go:293 +0x8c
github.com/sirupsen/logrus.(*Logger).Log(0x1305240, 0x0, {0xc00019bb80, 0x3, 0x3})
/home/user/Desktop/MapReduce/vendor/github.com/sirupsen/logrus/logger.go:198 +0x7a
github.com/sirupsen/logrus.(*Logger).Panic(...)
/home/user/Desktop/MapReduce/vendor/github.com/sirupsen/logrus/logger.go:247
github.com/sirupsen/logrus.Panic(...)
/home/user/Desktop/MapReduce/vendor/github.com/sirupsen/logrus/exported.go:129
github.com/BWbwchen/MapReduce/master.(*Master).availableWorkers(0xc000024870, 0x1)
/home/user/Desktop/MapReduce/master/master.go:149 +0x269
github.com/BWbwchen/MapReduce/master.(*Master).distributeMapTask(0xc000024870)
/home/user/Desktop/MapReduce/master/master.go:233 +0x60f
github.com/BWbwchen/MapReduce/master.StartMaster({0xc000078c40, 0x4, 0x4}, 0x0?, 0x0?, {0xc00001569a, 0x6})
/home/user/Desktop/MapReduce/master/mainInterface.go:47 +0x331
github.com/BWbwchen/MapReduce.startMaster.func1()
/home/user/Desktop/MapReduce/util.go:102 +0x8a
created by github.com/BWbwchen/MapReduce.startMaster in goroutine 1
/home/user/Desktop/MapReduce/util.go:101 +0x2a5
exit status 2
Originally posted by @kiarash8112 in #1 (comment)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.