hdfs - Fault Tolerance in Hadoop -


we know in hadoop if data corrupts new replica created if node down initally works fine how namenode deals 4 replicas delete 1 node?

if newly created 1 or 1 suddenly?

in situation when 1 data node goes down, name node see data blocks under-replicated , start replication other node in cluster bring replication expected level (default 3).

when corrupted node goes again blocks node seen over-replicated 4 replicas.

when block becomes over-replicated, name node chooses replica remove. name node prefer not reduce number of racks host replicas, , secondly prefer remove replica data node least amount of available disk space. may rebalancing load on cluster.


Comments

Post a Comment

Popular posts from this blog

gcc - MinGW's ld cannot perform PE operations on non PE output file -

i know there other similar questions out there, stackoverflow or not. i've researched lot this, , still didn't find single solution. i'm doing operative system side project. i've been doing in assembly, wanna join c code. test, made assembly code file (called test.asm): [bits 32] global _a section .text _a: jmp $ then made c file (called main.c): extern void a(void); int main(void) { a(); } to link, used file (called make.bat): "c:\mingw\bin\gcc.exe" -ffreestanding -c -o c.o main.c nasm -f coff -o asm.o test.asm "c:\mingw\bin\ld.exe" -ttext 0x100000 --oformat binary -o out.bin c.o asm.o pause i've been researching ages, , i'm still struggling find answer. hope won't flagged duplicate. acknowledge existence of similar questions, have different answers, , none work me. question: what doing wrong? old mingw versions had problem "ld" not able create non-pe files @ all. maybe current versions...
Read more

How to connect android app to App engine -

i'm trying connect app in android server i've opened on appengine. my server's name : dddd-daniel-2345 , website server : http://dddd-daniel-2345.appspot.com/ this how connect android app server in app engine: app.yaml: application: dddd-daniel-2345 version: 1 runtime: python27 api_version: 1 threadsafe: yes handlers: - url: .* script: main.app libraries: - name: webapp2 version: "2.5.2" main.py: #!/usr/bin/env python # # copyright 2007 google inc. google.appengine.api import users import webapp2 class userhandler(webapp2.requesthandler): def get(self): user = users.get_current_user() if user: greeting = ('%s,%s,%s' % (user.email(),user.user_id(),user.nickname())) else: greeting = ('not logged in') self.response.out.write(greeting) app = webapp2.wsgiapplication([('/', userhandler),], debug=true) my applications code : add manifast main....
Read more

hadoop - Running Map Reduce Job shows error - Mkdirs failed to create /var/folders/ -

when running map reduce job in mac terminal as: pawandeepsingh1$ hadoop jar maximumtemperature.jar exception in thread "main" java.io.ioexception: mkdirs failed create /var/folders/v1/lyx_f0rj615cy8s54_bk053h0000gp/t/hadoop-unjar3698429834837790177/meta-inf/license @ org.apache.hadoop.util.runjar.ensuredirectory(runjar.java:128) @ org.apache.hadoop.util.runjar.unjar(runjar.java:104) @ org.apache.hadoop.util.runjar.unjar(runjar.java:81) @ org.apache.hadoop.util.runjar.run(runjar.java:209) @ org.apache.hadoop.util.runjar.main(runjar.java:136) i have seen similar question says don't have permissions run job. can give me step wise solution. thank in advance you don't have permission on hdfs filesystem create dirs job. submitting cluster or running local testing environment. can su user has perms /var folders on hdfs? if aren't admin on cluster need have admin either add group ( hdfs, hadoop) have permissions on hdfs or give p...
Read more